Papers
Topics
Authors
Recent
Search
2000 character limit reached

How About Kind of Generating Hedges using End-to-End Neural Models?

Published 26 Jun 2023 in cs.CL and cs.AI | (2306.14696v1)

Abstract: Hedging is a strategy for softening the impact of a statement in conversation. In reducing the strength of an expression, it may help to avoid embarrassment (more technically, ``face threat'') to one's listener. For this reason, it is often found in contexts of instruction, such as tutoring. In this work, we develop a model of hedge generation based on i) fine-tuning state-of-the-art LLMs trained on human-human tutoring data, followed by ii) reranking to select the candidate that best matches the expected hedging strategy within a candidate pool using a hedge classifier. We apply this method to a natural peer-tutoring corpus containing a significant number of disfluencies, repetitions, and repairs. The results show that generation in this noisy environment is feasible with reranking. By conducting an error analysis for both approaches, we reveal the challenges faced by systems attempting to accomplish both social and task-oriented goals in conversation.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (68)
  1. Cognitive tutors: Lessons learned. The journal of the learning sciences, 4(2):167–207.
  2. Chris Berry and Allen Brizee. 2010. Identifying independent and dependent clauses. Purdue OWL.
  3. Longman grammar of spoken and written English, volume 2. Longman London.
  4. Enabling robots to understand indirect speech acts in task-based interactions. Journal of Human-Robot Interaction, 6(1):64–94.
  5. Gretchen P Brown. 1980. Characterizing indirect speech acts. American Journal of Computational Linguistics, 6(3-4):150–166.
  6. Penelope Brown and Stephen C. Levinson. 1987. Politeness: Some universals in language usage, volume 4. Cambridge university press.
  7. The teacher-student chatroom corpus. In Proceedings of the 9th Workshop on NLP for Computer Assisted Language Learning, pages 10–20.
  8. Justine Cassell. 2022. Socially interactive agents as peers. In The Handbook on Socially Interactive Agents: 20 years of Research on Embodied Conversational Agents, Intelligent Virtual Agents, and Social Robotics Volume 2: Interactivity, Platforms, Application, pages 331–366.
  9. Evaluation metrics for language models.
  10. Herbert H Clark. 1979. Responding to indirect speech acts. Cognitive psychology, 11(4):430–477.
  11. Automatic evaluation of end-to-end dialog systems with adequacy-fluency metrics. Computer Speech & Language, 55:200–215.
  12. M Robin DiMatteo. 1979. A social-psychological analysis of physician-patient rapport: toward a science of the art of medicine. Journal of Social Issues, 35(1):12–33.
  13. The second conversational intelligence challenge (convai2). In The NeurIPS’18 Competition: From Machine Learning to Intelligent Conversations, pages 187–208. Springer.
  14. ELI5: Long form question answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3558–3567, Florence, Italy. Association for Computational Linguistics.
  15. Bruce Fraser. 2010. Pragmatic competence: The case of hedging. new approaches to hedging.
  16. Rebecca A Glazier. 2016. Building rapport to improve retention and success in online classes. Journal of Political Science Education, 12(4):437–456.
  17. i think it might help if we multiply, and not add. In Detecting indirectness in conversation. In 9th International Workshop on Spoken Dialogue System Technology, page 27–40. Springer.
  18. Erving Goffman. 1967. Interaction Ritual, chapter On Face-Work. Pantheon, New York.
  19. Dwayne D Gremler and Kevin P Gwinner. 2008. Rapport-building behaviors used by retail employees. Journal of Retailing, 84(3):308–324.
  20. Simple and effective retrieve-edit-rerank text generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2532–2538, Online. Association for Computational Linguistics.
  21. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, 30.
  22. Klaus Krippendorff. 2004. Reliability in content analysis: Some common misconceptions and recommendations. Human communication research, 30(3):411–433.
  23. George Lakoff. 1975. Hedges: A study in meaning criteria and the logic of fuzzy concepts. In Contemporary research in philosophical logic and linguistic semantics, pages 221–271. Springer.
  24. Matthew J Leach. 2005. Rapport: A key to treatment success. Complementary therapies in clinical practice, 11(4):262–265.
  25. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871–7880, Online. Association for Computational Linguistics.
  26. DailyDialog: A manually labelled multi-turn dialogue dataset. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 986–995, Taipei, Taiwan. Asian Federation of Natural Language Processing.
  27. Incremental transformer with deliberation decoder for document grounded conversations. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 12–21.
  28. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  29. Towards emotional support dialog systems. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3469–3483, Online. Association for Computational Linguistics.
  30. Ilya Loshchilov and Frank Hutter. 2018. Decoupled weight decay regularization. In International Conference on Learning Representations.
  31. Politeness transfer: A tag and generate approach. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1869–1881, Online. Association for Computational Linguistics.
  32. The impact of peer tutors’ use of indirect feedback and instructions. Philadelphia, PA: International Society of the Learning Sciences.
  33. When to say what and how: Adapting the elaborateness and indirectness of spoken dialogue systems. Dialogue & Discourse, 13(1):1–40.
  34. Elizabeth Murphy and María A Rodríguez-Manzanares. 2012. Rapport in distance education. International Review of Research in Open and Distributed Learning, 13(1):167–190.
  35. Tong Niu and Mohit Bansal. 2018. Polite dialogue generation without parallel data. Transactions of the Association for Computational Linguistics, 6:373–389.
  36. OpenAI. 2022. Chatgpt: Optimizing language models for dialogue.
  37. Towards holistic and automatic evaluation of open-domain dialogue generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3619–3629.
  38. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
  39. C Raymond Perrault. 1980. A plan-based analysis of indirect speech act. American Journal of Computational Linguistics, 6(3-4):167–182.
  40. Brigitte Planken. 2005. Managing rapport in lingua franca sales negotiations: A comparison of professional and aspiring negotiators. English for Specific Purposes, 24(4):381–400.
  41. Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics.
  42. Kaśka Porayska-Pomsta and Chris Mellish. 2004. Modelling politeness in natural language generation. In International Conference on Natural Language Generation, pages 141–150. Springer.
  43. On hedging in physician-physician discourse. Linguistics and the Professions, 8(1):83–97.
  44. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  45. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas. Association for Computational Linguistics.
  46. ”You might think about slightly revising the title”: Identifying hedges in peer-tutoring interactions. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2160–2174, Dublin, Ireland. Association for Computational Linguistics.
  47. Towards empathetic open-domain conversation models: A new benchmark and dataset. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5370–5381.
  48. Recipes for building an open-domain chatbot. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 300–325, Online. Association for Computational Linguistics.
  49. Tim Rowland. 2007. well maybe not exactly, but it’s around fifty basically? In Vague language in mathematics classrooms. In Vague language explored, page 79–96. Springer.
  50. What makes a good conversation? how controllable attributes affect human judgments. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1702–1723.
  51. Enhancing self-disclosure in neural dialog models by candidate re-ranking. ArXiv preprint, abs/2109.05090.
  52. Helen Spencer-Oatey. 2005. (im)politeness, face and perceptions of rapport: Unpackaging their bases and interrelationships. 1(1):95–119.
  53. Lamda: Language models for dialog applications. ArXiv preprint, abs/2201.08239.
  54. Scott Thornbury and Diana Slade. 2006. Conversation: From description to pedagogy. Cambridge University Press.
  55. Linda Tickle-Degnen and Robert Rosenthal. 1990. The nature of rapport and its nonverbal correlates. Psychological inquiry, 1(4):285–293.
  56. Karen Tracy and Nikolas Coupland. 1990. Multiple goals in discourse: An overview of issues. Journal of Language and Social Psychology, 9(1-2):1–13.
  57. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998–6008.
  58. Veronika Vincze. 2014. Uncertainty detection in natural language texts. PhD, University of Szeged, 141.
  59. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112–1122, New Orleans, Louisiana. Association for Computational Linguistics.
  60. Timothy Williamson. 2002. Vagueness. Routledge.
  61. Bartscore: Evaluating generated text as text generation. Advances in Neural Information Processing Systems, 34.
  62. Personalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204–2213, Melbourne, Australia. Association for Computational Linguistics.
  63. Bertscore: Evaluating text generation with BERT. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
  64. DIALOGPT : Large-scale generative pre-training for conversational response generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 270–278, Online. Association for Computational Linguistics.
  65. Towards a dyadic computational model of rapport management for human-virtual agent interaction. In International conference on intelligent virtual agents, pages 514–527. Springer.
  66. Automatic recognition of conversational strategies in the service of a socially-aware dialog system. In Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 381–392, Los Angeles. Association for Computational Linguistics.
  67. Socially-aware virtual agents: Automatically assessing dyadic rapport from temporal patterns of behavior. In International conference on intelligent virtual agents, page 218–233. Springer.
  68. Inducing positive perspectives with text reframing. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3682–3700, Dublin, Ireland. Association for Computational Linguistics.
Citations (3)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up