政大機構典藏-National Chengchi University Institutional Repository(NCCUR):Item 140.119/107015

政大機構典藏-National Chengchi University Institutional Repository(NCCUR):Item 140.119/107015

English | 正體中文 | 简体中文 | Post-Print筆數 : 27 | 全文笔数/总笔数 : 114205/145239 (79%)
造访人次 : 52923813 在线人数 : 977

RC Version 6.0 © Powered By DSPACE, MIT. Enhanced by NTU Library IR team.

搜寻范围

查询小技巧：

您可在西文检索词汇前后加上"双引号"，以获取较精准的检索结果

若欲以作者姓名搜寻，建议至进阶搜寻限定作者字段，可获得较完整数据

进阶搜寻

主页 ‧ 登入 ‧ 上传 ‧ 说明 ‧ 关于政大典藏 ‧ 管理

到手机版

政大機構典藏 > 商學院 > 統計學系 > 學位論文 > Item 140.119/107015

请使用永久网址来引用或连结此文件: https://nccur.lib.nccu.edu.tw/handle/140.119/107015

题名:	詞彙向量的理論與評估基於矩陣分解與神經網絡 Theory and evaluation of word embedding based on matrix factorization and neural network
作者:	張文嘉 Jhang, Wun Jia
贡献者:	翁久幸馬偉雲 Weng, Chiu Hsing Ma, Wei Yun 張文嘉 Jhang, Wun Jia
关键词:	矩陣分解初始值自然語言處理神經網絡 Matrix factorization Initalization Natural language processing Neural network
日期:	2017
上传时间:	2017-03-02 11:10:02 (UTC+8)
摘要:	隨著機器學習在越來越多任務中有突破性的發展，特別是在自然語言處理問題上，得到越來越多的關注，近年來，詞向量是自然語言處理研究中最令人興奮的部分之一。在這篇論文中，我們討論了兩種主要的詞向量學習方法。一種是傳統的矩陣分解，如奇異值分解，另一種是基於神經網絡模型（具有負採樣的Skip-gram模型（Mikolov等人提出，2013），它是一種迭代演算法。我們提出一種方法來挑選初始值，透過使用奇異值分解得到的詞向量當作是Skip-gram模型的初始直，結果發現替換較佳的初始值，在某些自然語言處理的任務中得到明顯的提升。 Recently, word embedding is one of the most exciting part of research in natural language processing. In this thesis, we discuss the two major learning approaches for word embedding. One is traditional matrix factorization like singular value decomposition, the other is based on neural network model (e.g. the Skip-gram model with negative sampling (Mikolov et al., 2013b)) which is an iterative algorithm. It is known that an iterative process is sensitive to initial starting values. We present an approach for implementing the Skip-gram model with negative sampling from a given initial value that is using singular value decomposition. Furthermore, we show that refined initial starting points improve the analogy task and succeed in capturing fine-gained semantic and syntactic regularities using vector arithmetic.
參考文獻:	Marco Baroni, Georgiana Dinu, and German Kruszewski. Don`t count, predict! a systematic comparison of context-counting vs. context-predicting semantic vectors. In ACL (1), pages 238-247, 2014. Yoshua Bengio. Learning deep architectures for ai. Foundations and trends R in Machine Learning, 2(1):1-127, 2009. Christophe Biernacki, Gilles Celeux, and Gerard Govaert. Choosing starting values for the em algorithm for getting the highest likelihood in multivariate gaussian mixture models. Computational Statistics & Data Analysis, 41(3):561-575, 2003. Paul S Bradley and Usama M Fayyad. Refining initial points for k-means clustering. In ICML, volume 98, pages 91-99. Citeseer, 1998. Elia Bruni, Gemma Boleda, Marco Baroni, and Nam-Khanh Tran. Distributional semantics in technicolor. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pages 136-145. Association for Computational Linguistics, 2012. John Caron. Experiments with lsa scoring: Optimal rank and basis. In Proceedings of the SIAM Computational Information Retrieval Workshop, pages 157-169, 2001. Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillipp Koehn, and Tony Robinson. One billion word benchmark for measuring progress in statistical language modeling. arXiv preprint arXiv:1312.3005, 2013. Scott Deerwester, Susan T Dumais, George W Furnas, Thomas K Landauer, and Richard Harshman. Indexing by latent semantic analysis. Journal of the American society for information science, 41(6):391, 1990. Richard O Duda, Peter E Hart, et al. Pattern classification and scene analysis, volume 3. Wiley New York, 1973. Yoav Goldberg and Omer Levy. word2vec explained: deriving mikolov et al.`s negative-sampling word-embedding method. arXiv preprint arXiv:1402.3722, 2014. Felix Hill, Roi Reichart, and Anna Korhonen. Simlex-999: Evaluating semantic models with (genuine) similarity estimation. Computational Linguistics, 2016. Omer Levy and Yoav Goldberg. Dependency-based word embeddings. In ACL (2), pages 302-308, 2014a. Omer Levy and Yoav Goldberg. Neural word embedding as implicit matrix factorization.In Advances in Neural Information Processing Systems, pages 2177-2185, 2014b. Omer Levy, Yoav Goldberg, and Israel Ramat-Gan. Linguistic regularities in sparse and explicit word representations. In CoNLL, pages 171-180, 2014. Omer Levy, Yoav Goldberg, and Ido Dagan. Improving distributional similarity with lessons learned from word embeddings. Transactions of the Association for Computational Linguistics, 3:211-225, 2015. Thang Luong, Richard Socher, and Christopher D Manning. Better word representations with recursive neural networks for morphology. In CoNLL, pages 104-113, 2013. Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(Nov):2579-2605, 2008. Christopher D Manning, Prabhakar Raghavan, and Hinrich Schutze. Evaluation in information retrieval. Introduction to information retrieval, pages 151-175, 2008. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Ecient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013a. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111-3119, 2013b. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. Linguistic regularities in continuous space word representations. In HLT-NAACL, volume 13, pages 746-751, 2013c. Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In EMNLP, volume 14, pages 1532-43, 2014. Kira Radinsky, Eugene Agichtein, Evgeniy Gabrilovich, and Shaul Markovitch. A word at a time: computing word relatedness using temporal semantic analysis. In Proceedings of the 20th international conference on World wide web, pages 337-346. ACM, 2011. Xin Rong. word2vec parameter learning explained. arXiv preprint arXiv:1411.2738, 2014. Gerard Salton and Michael J McGill. Introduction to modern information retrieval. 1986. Fabrizio Sebastiani. Machine learning in automated text categorization. ACM computing surveys (CSUR), 34(1):1-47, 2002. Richard Socher, John Bauer, Christopher D Manning, and Andrew Y Ng. Parsing with compositional vector grammars. In ACL (1), pages 455-465, 2013a. Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the conference on empirical methods in natural language processing (EMNLP), volume 1631, page 1642. Citeseer, 2013b. Stefanie Tellex, Boris Katz, Jimmy Lin, Aaron Fernandes, and Gregory Marton. Quantitative evaluation of passage retrieval algorithms for question answering. In Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, pages 41-47. ACM, 2003. Joseph Turian, Lev Ratinov, and Yoshua Bengio. Word representations: a simple and general method for semi-supervised learning. In Proceedings of the 48th annual meeting of the association for computational linguistics, pages 384-394. Association for Computational Linguistics, 2010. Will Y Zou, Richard Socher, Daniel M Cer, and Christopher D Manning. Bilingual word embeddings for phrase-based machine translation. In EMNLP, pages 1393-1398, 2013.
描述:	碩士國立政治大學統計學系 103354027
資料來源:	http://thesis.lib.nccu.edu.tw/record/#G0103354027
数据类型:	thesis
显示于类别:	[統計學系] 學位論文

文件中的档案:

档案	大小	格式	浏览次数
402701.pdf	856Kb	Adobe PDF2	40	检视/开启

在政大典藏中所有的数据项都受到原著作权保护.

社群 sharing

著作權政策宣告 Copyright Announcement

1.本網站之數位內容為國立政治大學所收錄之機構典藏，無償提供學術研究與公眾教育等公益性使用，惟仍請適度，合理使用本網站之內容，以尊重著作權人之權益。商業上之利用，則請先取得著作權人之授權。
The digital content of this website is part of National Chengchi University Institutional Repository. It provides free access to academic research and public education for non-commercial use. Please utilize it in a proper and reasonable manner and respect the rights of copyright owners. For commercial use, please obtain authorization from the copyright owner in advance.

2.本網站之製作，已盡力防止侵害著作權人之權益，如仍發現本網站之數位內容有侵害著作權人權益情事者，請權利人通知本網站維護人員(nccur@nccu.edu.tw)，維護人員將立即採取移除該數位著作等補救措施。
NCCU Institutional Repository is made to protect the interests of copyright owners. If you believe that any material on the website infringes copyright, please contact our staff(nccur@nccu.edu.tw). We will remove the work from the repository and investigate your claim.

DSpace Software Copyright © 2002-2004 MIT & Hewlett-Packard / Enhanced by NTU Library IR team Copyright © - 回馈