政大機構典藏-National Chengchi University Institutional Repository(NCCUR):Item 140.119/152571

政大機構典藏-National Chengchi University Institutional Repository(NCCUR):Item 140.119/152571

English | 正體中文 | 简体中文 | Post-Print筆數 : 27 | Items with full text/Total items : 113648/144635 (79%)
Visitors : 51667905 Online Users : 546

RC Version 6.0 © Powered By DSPACE, MIT. Enhanced by NTU Library IR team.

Scope

please add "double quotation mark" for query phrases to get precise results

please goto advance search for comprehansive author search

Adv. Search

Home ‧ Login ‧ Upload ‧ Help ‧ About ‧ Administer

Goto mobile version

政大典藏 > College of Informatics > Department of Computer Science > Theses > Item 140.119/152571

Please use this identifier to cite or link to this item: https://nccur.lib.nccu.edu.tw/handle/140.119/152571

Title:	基於 LLM 的無監督多顆粒度重排序用於長文本檢索 Unsupervised Multi-granularity LLM-based Reranking for Long Text Retrieval
Authors:	吳家瑋 Wu, Chia-Wei
Contributors:	李蔡彥黃瀚萱 Li, Tsai-Yen Huang, Hen-Hsen 吳家瑋 Wu, Chia-Wei
Keywords:	資訊檢索大型語言模型查詢重寫文本壓縮長文本無監督式文本重新排序 Information Retrieval Large Language Model Query Rewriting Text Compression Long Text Unsupervised Text Reranking
Date:	2024
Issue Date:	2024-08-05 12:45:39 (UTC+8)
Abstract:	本研究提出Rate and Rank GPT（RRGPT），以提高文本重排序的效能與效率，並解決使用大型語言模型進行文檔檢索任務時遇到的長文本挑戰。RRGPT是一種新穎的資訊檢索方法，利用大型語言模型輔助資訊檢索系統中的子任務：查詢重寫任務和無監督式文本重新排序任務。在查詢重寫任務中，本研究將大型語言模型產生的關鍵術語堆疊起來，以擴充原始查詢。在無監督文本重新排序任務中，本研究提出混合式文本重新排序演算法，透過多顆粒度和低成本的方式，依相關度重新排序文本列表。對於長文本問題，本研究採用文本壓縮法從長文本中提取關鍵訊息，以確保文本符合大型語言模型的輸入長度限制。最後，本研究使用DL19和DL20的資料集驗證RRGPT在文檔檢索任務和段落檢索任務的表現。結果表明，RRGPT能更好地依相關度重排序文本列表，並且解決長文本問題。 This research proposes Rate and Rank GPT (RRGPT) to enhance the effectiveness and efficiency of text reranking and to address the challenges associated with long text in document retrieval tasks using Large Language Models (LLMs). RRGPT is a novel information retrieval method that utilize LLMs to improve subtasks such as query rewriting and unsupervised text reranking within the information retrieval system. For the query rewriting task, this research stacks terms generated by LLMs to expand queries. For the unsupervised text reranking task, this research proposes the hybrid text reranking algorithm with multi-granularity that ranks a list of texts with higher accuracy and lower cost than traditional methods. For the long text issue, this research uses a text compression strategy to extract crucial information from long texts, ensuring the texts compliance the input length constraints of LLMs. Finally, this research empirically validate the effectiveness and efficiency of RRGPT using the DL19 and DL20 datasets for document retrieval tasks and passage retrieval tasks. The empirical results demonstrate that RRPGT improves the effectiveness and efficiency text reranking and addresses long text issue.
Reference:	[1] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. [2] Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Zhicheng Dou, and Ji-Rong Wen. Large language models for information retrieval: A survey. arXiv preprint arXiv:2308.07107, 2023. [3] Weiwei Sun, Lingyong Yan, Xinyu Ma, Pengjie Ren, Dawei Yin, and Zhaochun Ren. Is chatgpt good at search? investigating large language models as re-ranking agent. arXiv preprint arXiv:2304.09542, 2023. [4] Rolf Jagerman, Honglei Zhuang, Zhen Qin, Xuanhui Wang, and Michael Bendersky. Query expansion by prompting large language models. arXiv preprint arXiv:2305.03653, 2023. [5] Honglei Zhuang, Zhen Qin, Kai Hui, Junru Wu, Le Yan, Xuanhui Wang, and Michael Berdersky. Beyond yes and no: Improving zero-shot llm rankers via scoring finegrained relevance labels. arXiv preprint arXiv:2310.14122, 2023. [6] Big Data, Big Impact: New Possibilities for International Development. (2012, January 22). World Economic Forum. [7] Steve Loh. (2012, February 11). The Age of Big Data. The New York Times. [8] Martin Hilbert and Priscila López. The world’s technological capacity to store, communicate, and compute information. science, 332(6025):60–65, 2011. [9] Netcraft. August 2011 Web Server Survey. https://web.archive.org/, 2011. [10] The Impact of TV in the U.S. Daily Iowan. 22 Nov 1955: 2. [11] OpenAI. OpenAI: Introducing ChatGPT. https://openai.com/, 2022. [12] Google. Google: Introducing Gemini. https://ai.google.dev/, 2023. [13] Alonzo Church. George boole. an investigation of the laws of thought, on which are founded the mathematical theories of logic and probabilities. dover publications, inc., new york1951, 11+ 424 pp. The Journal of Symbolic Logic, 16(3):224–225, 1951. [14] Gerard Salton, Anita Wong, and Chung-Shu Yang. A vector space model for automatic indexing. Communications of the ACM, 18(11):613–620, 1975. [15] Xiaoyong Liu and W Bruce Croft. Statistical language modeling for information retrieval. Annu. Rev. Inf. Sci. Technol., 39(1):1–31, 2005. [16] Fei Song and W Bruce Croft. A general language model for information retrieval. In Proceedings of the eighth international conference on Information and knowledge management, pages 316–321, 1999. [17] David RH Miller, Tim Leek, and Richard M Schwartz. A hidden markov model information retrieval system. In Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, pages 214–221, 1999. [18] Stephen E Robertson and K Sparck Jones. Relevance weighting of search terms. Journal of the American Society for Information science, 27(3):129–146, 1976. [19] Wayne Xin Zhao, Jing Liu, Ruiyang Ren, and Ji-Rong Wen. Dense text retrieval based on pretrained language models: A survey. ACM Transactions on Information Systems, 42(4):1–60, 2024. [20] Jiafeng Guo, Yixing Fan, Qingyao Ai, and W Bruce Croft. A deep relevance matching model for ad-hoc retrieval. In Proceedings of the 25th ACM international on conference on information and knowledge management, pages 55–64, 2016. [21] Qingyao Ai, Ting Bai, Zhao Cao, Yi Chang, Jiawei Chen, Zhumin Chen, Zhiyong Cheng, Shoubin Dong, Zhicheng Dou, Fuli Feng, et al. Information retrieval meets large language models: a strategic report from chinese ir community. AI Open, 4:80–90, 2023. [22] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020. [23] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. [24] Luciano Floridi and Massimo Chiriatti. Gpt-3: Its nature, scope, limits, and consequences. Minds and Machines, 30:681–694, 2020. [25] Gerard Salton and Chris Buckley. Improving retrieval performance by relevance feedback. Journal of the American society for information science, 41(4):288–297, 1990. [26] Ian Ruthven and Mounia Lalmas. A survey on the use of relevance feedback for information access systems. The Knowledge Engineering Review, 18(2):95–145, 2003. [27] Guihong Cao, Jian-Yun Nie, Jianfeng Gao, and Stephen Robertson. Selecting good expansion terms for pseudo-relevance feedback. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, pages 243–250, 2008. [28] Yuanhua Lv and ChengXiang Zhai. Positional relevance model for pseudo-relevance feedback. In Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval, pages 579–586, 2010. [29] Kevyn Collins-Thompson and Jamie Callan. Estimation and use of uncertainty in pseudo-relevance feedback. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pages 303–310, 2007. [30] Rong Yan, Alexander Hauptmann, and Rong Jin. Multimedia search with pseudorelevance feedback. In Image and Video Retrieval: Second International Conference, CIVR 2003 Urbana-Champaign, IL, USA, July 24–25, 2003 Proceedings 2, pages 238–247. Springer, 2003. [31] Jiazhan Feng, Chongyang Tao, Xiubo Geng, Tao Shen, Can Xu, Guodong Long, Dongyan Zhao, and Daxin Jiang. Knowledge refinement via interaction between search engines and large language models. arXiv preprint arXiv:2305.07402, 2023. [32] Liang Wang, Nan Yang, and Furu Wei. Query2doc: Query expansion with large language models. corr abs/2303.07678 (2023), 2023. [33] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824– 24837, 2022. [34] Wenjun Peng, Guiyang Li, Yue Jiang, Zilong Wang, Dan Ou, Xiaoyi Zeng, Enhong Chen, et al. Large language model based long-tail query rewriting in taobao search. arXiv preprint arXiv:2311.03758, 2023. [35] Krishna Srinivasan, Karthik Raman, Anupam Samanta, Lingrui Liao, Luca Bertelli, and Mike Bendersky. Quill: Query intent with large language models using retrieval augmentation and multi-stage distillation. arXiv preprint arXiv:2210.15718, 2022. [36] Ellen M Voorhees et al. Overview of the trec 2003 robust retrieval track. In Trec, pages 69–77, 2003. [37] Kirk Roberts, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, Kyle Lo, Ian Soboroff, Ellen Voorhees, Lucy Lu Wang, and William R Hersh. Searching for scientific evidence in a pandemic: An overview of trec-covid. Journal of Biomedical Informatics, 121:103865, 2021. [38] Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M Voorhees. Overview of the trec 2019 deep learning track. arXiv preprint arXiv:2003.07820, 2020. [39] Xueguang Ma, Xinyu Zhang, Ronak Pradeep, and Jimmy Lin. Zero-shot listwise document reranking with a large language model. arXiv preprint arXiv:2305.02156, 2023. [40] Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, Xuanhui Wang, et al. Large language models are effective text rankers with pairwise ranking prompting. arXiv preprint arXiv:2306.17563, 2023. [41] Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, and Rodrigo Nogueira. Pyserini: A python toolkit for reproducible information retrieval research with sparse and dense representations. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2356–2362, 2021. [42] Peilin Yang, Hui Fang, and Jimmy Lin. Anserini: Reproducible ranking baselines using lucene. Journal of Data and Information Quality (JDIQ), 10(4):1–20, 2018. [43] Peilin Yang, Hui Fang, and Jimmy Lin. Anserini: Enabling the use of lucene for information retrieval research. In Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, pages 1253–1256, 2017. [44] Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04906, 2020. [45] Emine Yilmaz Daniel Campos Ellen M. Voorhees Nick Craswell, Bhaskar Mitra. Overview of the trec 2020 deep learning track. arXiv preprint arXiv:2102.07662, 2021. [46] Christophe Van Gysel and Maarten de Rijke. Pytrec_eval: An extremely fast python interface to trec_eval. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 873–876, 2018. [47] Monika Arora, Uma Kanjilal, and Dinesh Varshney. Evaluation of information retrieval: precision and recall. International Journal of Indian Culture and Business Management, 12(2):224–236, 2016. [48] Tie-Yan Liu et al. Learning to rank for information retrieval. Foundations and Trends® in Information Retrieval, 3(3):225–331, 2009. [49] Kazuaki Kishida. Property of average precision and its generalization: An examination of evaluation indicator for information retrieval experiments. National Institute of Informatics Tokyo, Japan, 2005. [50] Kalervo Järvelin and Jaana Kekäläinen. Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems (TOIS), 20(4):422–446, 2002. [51] Lin CY Rouge. A package for automatic evaluation of summaries. In Proceedings of Workshop on Text Summarization of ACL, Spain, volume 5, 2004. [52] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675,2019. [53] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pretraining of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
Description:	碩士國立政治大學資訊科學系 111753141
Source URI:	http://thesis.lib.nccu.edu.tw/record/#G0111753141
Data Type:	thesis
Appears in Collections:	[Department of Computer Science ] Theses

Files in This Item:

File	Description	Size	Format
314101.pdf		1583Kb	Adobe PDF	0	View/Open

All items in 政大典藏 are protected by copyright, with all rights reserved.

社群 sharing

著作權政策宣告 Copyright Announcement

1.本網站之數位內容為國立政治大學所收錄之機構典藏，無償提供學術研究與公眾教育等公益性使用，惟仍請適度，合理使用本網站之內容，以尊重著作權人之權益。商業上之利用，則請先取得著作權人之授權。
The digital content of this website is part of National Chengchi University Institutional Repository. It provides free access to academic research and public education for non-commercial use. Please utilize it in a proper and reasonable manner and respect the rights of copyright owners. For commercial use, please obtain authorization from the copyright owner in advance.

2.本網站之製作，已盡力防止侵害著作權人之權益，如仍發現本網站之數位內容有侵害著作權人權益情事者，請權利人通知本網站維護人員(nccur@nccu.edu.tw)，維護人員將立即採取移除該數位著作等補救措施。
NCCU Institutional Repository is made to protect the interests of copyright owners. If you believe that any material on the website infringes copyright, please contact our staff(nccur@nccu.edu.tw). We will remove the work from the repository and investigate your claim.

DSpace Software Copyright © 2002-2004 MIT & Hewlett-Packard / Enhanced by NTU Library IR team Copyright © - Feedback