Papers
Topics
Authors
Recent
Search
2000 character limit reached

Fisher-Weighted Merge of Contrastive Learning Models in Sequential Recommendation

Published 5 Jul 2023 in cs.IR, cs.AI, and cs.LG | (2307.05476v1)

Abstract: Along with the exponential growth of online platforms and services, recommendation systems have become essential for identifying relevant items based on user preferences. The domain of sequential recommendation aims to capture evolving user preferences over time. To address dynamic preference, various contrastive learning methods have been proposed to target data sparsity, a challenge in recommendation systems due to the limited user-item interactions. In this paper, we are the first to apply the Fisher-Merging method to Sequential Recommendation, addressing and resolving practical challenges associated with it. This approach ensures robust fine-tuning by merging the parameters of multiple models, resulting in improved overall performance. Through extensive experiments, we demonstrate the effectiveness of our proposed methods, highlighting their potential to advance the state-of-the-art in sequential learning and recommendation systems.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (26)
  1. Breiman, L. Bagging predictors. Machine learning, 24:123–140, 1996.
  2. Breiman, L. Random forests. Machine learning, 45:5–32, 2001.
  3. Laplace redux-effortless bayesian deep learning. Advances in Neural Information Processing Systems, 34:20089–20103, 2021.
  4. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
  5. Ensemble deep learning: A review. Engineering Applications of Artificial Intelligence, 115:105151, 2022.
  6. Beyond accuracy: quantifying trial-by-trial behaviour of cnns and humans by measuring error consistency. Advances in Neural Information Processing Systems, 33:13890–13902, 2020.
  7. No one representation to rule them all: Overlapping features of training methods. arXiv preprint arXiv:2110.12899, 2021.
  8. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis), 5(4):1–19, 2015.
  9. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web, pp.  173–182, 2017.
  10. Parameter-efficient transfer learning for nlp. In International Conference on Machine Learning, pp. 2790–2799. PMLR, 2019.
  11. Self-attentive sequential recommendation. In 2018 IEEE international conference on data mining (ICDM), pp.  197–206. IEEE, 2018.
  12. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521–3526, 2017.
  13. Facial expression recognition via a boosted deep belief network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  1805–1812, 2014.
  14. MacKay, D. J. A practical bayesian framework for backpropagation networks. Neural computation, 4(3):448–472, 1992.
  15. Merging models with fisher-weighted averaging. Advances in Neural Information Processing Systems, 35:17703–17716, 2022.
  16. Deep ensembles for low-data transfer learning. arXiv preprint arXiv:2010.06866, 2020.
  17. Gradient boosting machines, a tutorial. Frontiers in neurorobotics, 7:21, 2013.
  18. Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems, 32, 2019.
  19. Contrastive learning for representation degeneration problem in sequential recommendation. In Proceedings of the fifteenth ACM international conference on web search and data mining, pp.  813–823, 2022.
  20. Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management, pp.  1441–1450, 2019.
  21. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International Conference on Machine Learning, pp. 9929–9939. PMLR, 2020.
  22. Hyperparameter ensembles for robustness and uncertainty quantification. Advances in Neural Information Processing Systems, 33:6514–6527, 2020.
  23. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In International Conference on Machine Learning, pp. 23965–23998. PMLR, 2022.
  24. Contrastive learning for sequential recommendation. In 2022 IEEE 38th international conference on data engineering (ICDE), pp.  1259–1273. IEEE, 2022.
  25. Understanding neural networks through deep visualization. arXiv preprint arXiv:1506.06579, 2015.
  26. Neural ensemble search for uncertainty estimation and dataset shift. Advances in Neural Information Processing Systems, 34:7898–7911, 2021.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up