KAHANI: Culturally-Nuanced Visual Storytelling Tool for Non-Western Cultures

Abstract: LLMs and Text-To-Image (T2I) models have demonstrated the ability to generate compelling text and visual stories. However, their outputs are predominantly aligned with the sensibilities of the Global North, often resulting in an outsider's gaze on other cultures. As a result, non-Western communities have to put extra effort into generating culturally specific stories. To address this challenge, we developed a visual storytelling tool called Kahani that generates culturally grounded visual stories for non-Western cultures. Our tool leverages off-the-shelf models GPT-4 Turbo and Stable Diffusion XL (SDXL). By using Chain of Thought (CoT) and T2I prompting techniques, we capture the cultural context from user's prompt and generate vivid descriptions of the characters and scene compositions. To evaluate the effectiveness of Kahani, we conducted a comparative user study with ChatGPT-4 (with DALL-E3) in which participants from different regions of India compared the cultural relevance of stories generated by the two tools. The results of the qualitative and quantitative analysis performed in the user study show that Kahani's visual stories are more culturally nuanced than those generated by ChatGPT-4. In 27 out of 36 comparisons, Kahani outperformed or was on par with ChatGPT-4, effectively capturing cultural nuances and incorporating more Culturally Specific Items (CSI), validating its ability to generate culturally grounded visual stories.

• The paper introduces a pipeline that leverages GPT-4 Turbo and SDXL to generate visually detailed, culturally authentic narratives.
• It employs a methodology combining cultural context extraction and Chain of Thought prompting, outperforming ChatGPT-4 in 27 out of 36 evaluations.
• The study underscores implications for inclusive education and AI, setting a new standard for culturally nuanced digital storytelling despite its limited focus on Indian contexts.

An Analysis of Culturally Nuanced Visual Storytelling for Non-Western Cultures

The study presented by the authors explores a culturally nuanced visual storytelling pipeline designed to generate culturally specific stories for non-Western communities. The pipeline, which utilizes the advanced capabilities of GPT-4 Turbo and Stable Diffusion XL (SDXL), aims to mitigate the influence of Western sensibilities prevalent in conventional LLMs and T2I models. The research addresses a significant gap by enhancing the cultural vibrancy and accuracy in automated storytelling, which is crucial given the rising global emphasis on cultural diversity in digital content.

In evaluating the storytelling pipeline, the researchers employed a comparative user study involving participants from various Indian regions. This study showcased that the pipeline's output characteristically includes more Culturally Specific Items (CSIs) than existing tools like ChatGPT-4. The qualitative and quantitative measures employed confirm the pipeline's superiority in 27 out of 36 evaluations regarding cultural competence and story generation quality, signaling an advancement in producing culturally relevant narratives for non-Western audiences.

The study highlights several methodological insights. The pipeline includes critical steps from extracting cultural context and writing stories to generating visuals, with careful attention to cultural details, leveraging Chain of Thought (CoT), and specific prompting techniques. This attention to methodology ensures alignment of the narrative content to cultural contexts, thus enhancing the generated stories' authenticity. For instance, scenes are meticulously planned to reflect real-life scenarios, including geographically accurate elements, and character descriptions focus on visual aspects significant to respective cultures without succumbing to generic archetypes.

In terms of implications, the study paves the way for more inclusive forms of visual storytelling. Practically, such advancements can impact education by providing culturally relatable content that could improve engagement and learning outcomes. Theoretically, this work challenges existing models to enhance their outputs in cultural representation, calling for a future where AI-generated content acknowledges and reflects diverse cultural narratives.

Considering the limitations and potential future directions, the study only explores a limited cultural spectrum within India. Expanding this research globally would offer richer insights into adapting AI storytelling tools for broader non-Western narratives. Additionally, refining the generation process to limit stereotypes while maintaining diversity and examining iterative feedback scenarios will bolster storytelling accuracy and inclusivity.

Overall, this research makes significant strides in culturally informed AI development, emphasizing the importance of aligning technological advancements with diverse cultural contexts. By addressing representational biases, the pipeline sets a new standard in AI storytelling, encouraging future developments to incorporate cultural nuances in a way that resonates with global audiences.