Stability Analysis of ChatGPT-based Sentiment Analysis in AI Quality Assurance
Abstract: In the era of large AI models, the complex architecture and vast parameters present substantial challenges for effective AI quality management (AIQM), e.g. LLM. This paper focuses on investigating the quality assurance of a specific LLM-based AI product--a ChatGPT-based sentiment analysis system. The study delves into stability issues related to both the operation and robustness of the expansive AI model on which ChatGPT is based. Experimental analysis is conducted using benchmark datasets for sentiment analysis. The results reveal that the constructed ChatGPT-based sentiment analysis system exhibits uncertainty, which is attributed to various operational factors. It demonstrated that the system also exhibits stability issues in handling conventional small text attacks involving robustness.
- Machine Learning Quality Management Guideline(https://www.digiarc.aist.go.jp/en/publication/aiqm/)(https://www.digiarc.aist.go.jp/en/publication/aiqm/)( italic_h italic_t italic_t italic_p italic_s : / / italic_w italic_w italic_w . italic_d italic_i italic_g italic_i italic_a italic_r italic_c . italic_a italic_i italic_s italic_t . italic_g italic_o . italic_j italic_p / italic_e italic_n / italic_p italic_u italic_b italic_l italic_i italic_c italic_a italic_t italic_i italic_o italic_n / italic_a italic_i italic_q italic_m / )
- OpenAI. https://chat.openai.com.chathttps://chat.openai.com.chatitalic_h italic_t italic_t italic_p italic_s : / / italic_c italic_h italic_a italic_t . italic_o italic_p italic_e italic_n italic_a italic_i . italic_c italic_o italic_m . italic_c italic_h italic_a italic_t, 2023.
- https://community.openai.com/t/chatgpt−results−much−better−than−api/336749https://community.openai.com/t/chatgpt-results-much-better-than-api/336749italic_h italic_t italic_t italic_p italic_s : / / italic_c italic_o italic_m italic_m italic_u italic_n italic_i italic_t italic_y . italic_o italic_p italic_e italic_n italic_a italic_i . italic_c italic_o italic_m / italic_t / italic_c italic_h italic_a italic_t italic_g italic_p italic_t - italic_r italic_e italic_s italic_u italic_l italic_t italic_s - italic_m italic_u italic_c italic_h - italic_b italic_e italic_t italic_t italic_e italic_r - italic_t italic_h italic_a italic_n - italic_a italic_p italic_i / 336749
- https://community.openai.com/t/different−output−generated−for−same−prompt−in−chat−mode−and−api−mode−using−gpt−3−5−turbo/318246https://community.openai.com/t/different-output-generated-for-same-prompt-in-% chat-mode-and-api-mode-using-gpt-3-5-turbo/318246italic_h italic_t italic_t italic_p italic_s : / / italic_c italic_o italic_m italic_m italic_u italic_n italic_i italic_t italic_y . italic_o italic_p italic_e italic_n italic_a italic_i . italic_c italic_o italic_m / italic_t / italic_d italic_i italic_f italic_f italic_e italic_r italic_e italic_n italic_t - italic_o italic_u italic_t italic_p italic_u italic_t - italic_g italic_e italic_n italic_e italic_r italic_a italic_t italic_e italic_d - italic_f italic_o italic_r - italic_s italic_a italic_m italic_e - italic_p italic_r italic_o italic_m italic_p italic_t - italic_i italic_n - italic_c italic_h italic_a italic_t - italic_m italic_o italic_d italic_e - italic_a italic_n italic_d - italic_a italic_p italic_i - italic_m italic_o italic_d italic_e - italic_u italic_s italic_i italic_n italic_g - italic_g italic_p italic_t - 3 - 5 - italic_t italic_u italic_r italic_b italic_o / 318246
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.