Loading...
|
Please use this identifier to cite or link to this item:
https://nccur.lib.nccu.edu.tw/handle/140.119/152565
|
| Title: | 適應性新聞生成：整合大型語言模型和圖神經網路以模仿新聞媒體立場風格
Adaptive News Generation: Integrating large language models and graph neural networks to mimic newspaper stances |
| Authors: | 彭怡靜
Peng, Yi-Ching |
| Contributors: | 李蔡彥
黃瀚萱
Li, Tsai-Yen
Huang, Hen-Hsen
彭怡靜
Peng, Yi-Ching |
| Keywords: | 大型語言模型
圖神經網路
自動新聞學
文本生成
自然語言生成
Large Language Model
Graph Neural Network
Automated Journalism
Text Generation
Natural Language Generation |
| Date: | 2024 |
| Issue Date: | 2024-08-05 12:44:28 (UTC+8) |
| Abstract: | 每日國內、外皆有大量事件發生，新聞媒體需要在一天內快速發布新聞以傳遞最新的資訊，大量新聞發布的需求使得新聞媒體需要耗費大量成本產生報導。隨著自然語言處理（Natural Language Processing, NLP）的技術快速發展，相關的應用也廣泛應用在各個領域，像是美聯社、華盛頓郵報等國外報社已應用自動新聞學（Automated Journalism）來幫助新聞的生成，但此類自動新聞學的方法需要依靠結構化的表格，例如財報、得分紀錄等，才可以幫助新聞生成。本研究希望可以降低新聞媒體產出新聞的成本，將媒體發布的新聞內容及其他屬性建構成圖，並以圖神經網路（Graph Neural Network, GNN）的概念透過大型語言模型（Large Language Model, LLM）的幫助來生成新聞，生成符合事實且文句通暢的新聞。本研究證實透過我們的方法可以生成人、事、地及寫作風格都接近目標報社的新聞，並且兼顧了初步新聞生成的速度及品質；我們也設計了Systemized Auto Prompt Engineering的新方法，以自動化調整包含多個文字提示的系統。
Daily, numerous events occur both domestically and internationally, requiring media outlets to quickly disseminate news. This demand imposes significant costs on news organizations. With the rapid advancement of Natural Language Processing (NLP) technology, its applications have expanded widely. Newspapers like the Associated Press and The Washington Post have implemented Automated Journalism to facilitate news generation. However, current methods in automated journalism rely on structured data tables, such as financial reports and score records.
This study aims to mitigate the cost associated with news production by structuring media-released news content and other attributes into a graph. Leveraging the concepts of Graph Neural Networks (GNNs) and the assistance of Large Language Models (LLMs), we seek to generate news efficiently without relying on structured tables, thereby contributing to a more cost-effective news production process.
This study confirms that our method can generate news articles closely matching the target media in terms of people, events, locations, and style, balancing speed and quality. Additionally, we designed Systemized Auto Prompt Engineering to automate system adjustments with multiple text prompts. |
| Reference: | [1] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.
[2] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[3] Ratish Puduppully, Li Dong, and Mirella Lapata. Data-to-text generation with content selection and planning. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 6908–6915, 2019.
[4] Saurabh Gupta, Huy H Nguyen, Junichi Yamagishi, and Isao Echizen. Viable threat on news reading: Generating biased news using natural language models. arXiv preprint arXiv:2010.02150, 2020.
[5] Wei-Fan Chen, Henning Wachsmuth, Khalid Al Khatib, and Benno Stein. Learning to flip the bias of news headlines. In Proceedings of the 11th International conference on natural language generation, pages 79–88, 2018.
[6] Yijun Tian, Huan Song, Zichen Wang, Haozhu Wang, Ziqing Hu, Fang Wang, Nitesh V Chawla, and Panpan Xu. Graph neural prompting with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19080–19088, 2024.
[7] Zemin Liu, Xingtong Yu, Yuan Fang, and Xinming Zhang. Graphprompt: Unifying pre-training and downstream tasks for graph neural networks. In Proceedings of the ACM Web Conference 2023, pages 417–428, 2023.
[8] Ramy Baly, Giovanni Da San Martino, James Glass, and Preslav Nakov. We can detect your bias: Predicting the political ideology of news articles. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4982–4991, 2020.
[9] Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36, 2024.
[10] Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910, 2022.
[11] Ramy Baly, Georgi Karadzhov, Dimitar Alexandrov, James Glass, and Preslav Nakov. Predicting factuality of reporting and bias of news media sources. arXiv preprint arXiv:1810.01765, 2018.
[12] Ramy Baly, Georgi Karadzhov, Jisun An, Haewoon Kwak, Yoan Dinkov, Ahmed Ali, James Glass, and Preslav Nakov. What was written vs. who read it: News media profiling using text analysis and social media context. arXiv preprint arXiv:2005.04518, 2020.
[13] Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. 2018.
[14] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
[15] Matt Carlson. The robotic reporter: Automated journalism and the redefinition of labor, compositional forms, and journalistic authority. In Journalism in an Era of Big Data, pages 108–123. Routledge, 2018.
[16] Carl-Gustav Linden. Decades of automation in the newsroom: Why are there still so many jobs in journalism? Digital journalism, 5(2):123–140, 2017.
[17] Leo Leppänen, Myriam Munezero, Mark Granroth-Wilding, and Hannu Toivonen. Data-driven news generation for automated journalism. In Proceedings of the 10th international conference on natural language generation, pages 188–197, 2017.
[18] Shyi-Ming Chen and Ming-Hung Huang. Automatically generating the weather news summary based on fuzzy reasoning and ontology techniques. Information Sciences, 279:746–763, 2014.
[19] Liubov Nesterenko. Building a system for stock news generation in russian. In Proceedings of the 2nd international workshop on natural language generation and the semantic web (webnlg 2016), pages 37–40, 2016.
[20] Sheila Mendez-Nunez and Gracian Trivino. Combining semantic web technologies and computational theory of perceptions for text generation in financial analysis. In International Conference on Fuzzy Systems, pages 1–8. IEEE, 2010.
[21] Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. Defending against neural fake news. Advances in neural information processing systems, 32, 2019.
[22] Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: A review of methods and applications. AI open, 1:57–81, 2020.
[23] Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, et al. Gpt-neox-20b: An open-source autoregressive language model. arXiv preprint arXiv:2204.06745, 2022.
[24] Stephen Robertson, Hugo Zaragoza, et al. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends® in Information Retrieval, 3(4):333–389, 2009.
[25] Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81, 2004.
[26] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675,
2019.
[27] Elad Levi, Eli Brosh, and Matan Friedmann. Intent-based prompt calibration: Enhancing prompt optimization with synthetic boundary cases. arXiv preprint
arXiv:2402.03099, 2024. |
| Description: | 碩士
國立政治大學
資訊科學系
111753106 |
| Source URI: | http://thesis.lib.nccu.edu.tw/record/#G0111753106 |
| Data Type: | thesis |
| Appears in Collections: | [資訊科學系] 學位論文
|
Files in This Item:
| File |
Description |
Size | Format | |
|---|
| 310601.pdf | | 4603Kb | Adobe PDF | 0 | View/Open |
|
All items in 政大典藏 are protected by copyright, with all rights reserved.
|
