政大機構典藏-National Chengchi University Institutional Repository(NCCUR):Item 140.119/124705
English  |  正體中文  |  简体中文  |  Post-Print筆數 : 27 |  全文筆數/總筆數 : 113160/144130 (79%)
造訪人次 : 50752076      線上人數 : 793
RC Version 6.0 © Powered By DSPACE, MIT. Enhanced by NTU Library IR team.
搜尋範圍 查詢小技巧：
  • 您可在西文檢索詞彙前後加上"雙引號"，以獲取較精準的檢索結果
  • 若欲以作者姓名搜尋，建議至進階搜尋限定作者欄位，可獲得較完整資料
    •  進階搜尋
    政大機構典藏 > 商學院 > 資訊管理學系 > 學位論文 >  Item 140.119/124705


    請使用永久網址來引用或連結此文件: https://nccur.lib.nccu.edu.tw/handle/140.119/124705


    題名: HiSeqGAN: 高維資料的序列合成與預測
    HiSeqGAN: High-dimensional Sequence Synthesis and Prediction
    作者: 田韻杰
    Tien, Yun-Chieh
    貢獻者: 郁方
    Yu, Fang
    田韻杰
    Tien, Yun-Chieh
    關鍵詞: 高維資料
    序列合成
    序列預測
    增長層級式自我組織映射圖
    序列對抗生成網路
    High-dimensional data
    Sequence Synthesis
    Sequence Prediction
    SeqGAN
    GHSOM
    日期: 2019
    上傳時間: 2019-08-07 16:05:57 (UTC+8)
    摘要: 隨著大數據時代來臨，許多資料都具有高維度的變數，而要如何提升高維度序列資料的預測準確度是重要的課題之一。本篇論文即是結合了深度學習的技術，針對高維度資料的序列提出一個有效合成和預測的新方法：HiSeqGAN。
    首先，我們會利用增長層級式自我組織映射圖(GHSOM)為資料進行結構化分群，接著透過編碼演算法將分群後的資料轉換成座標向量，給予高維度資料一個新的編碼方式。並採用序列對抗生成網路(SeqGAN)作為主要的訓練模型，以此合成和預測結構化資料的序列。
    本篇論文的貢獻在於利用序列對抗生成網路對結構化資料進行合成及預測，除了可以提供更多的資料用來訓練一個性能較好的遞歸神經網路 (RNN)模型之外，也能有效的提升預測結構化資料的準確性。
    High-dimensional data sequences constantly appear in practice. State-of-the-art models such as recurrent neural networks suffer prediction accuracy from complex relations among values of attributes. Adopting unsupervised clustering that clusters data based on their attribute value similarity results data in lower dimensions that can be structured in a hierarchical relation. It is essential to consider these data relations to improve the performance of training models. In this work, we propose a new approach to synthesize and predict sequences of data that are structured in a hierarchy. Specifically, we adopt a new hierarchical data encoding and seamlessly modify loss functions of SeqGAN as our training model to synthesize data sequences. In practice, we first use the hierarchical clustering algorithm, GHSOM, to cluster our training data. By relabelling a sample with the cluster that it falls to, we are able to use the GHSOM map to identify the hierarchical relation of samples. We then converse the clusters to the coordinate vectors with our hierarchical data encoding algorithm and replace the loss function with maximizing cosine similarity in the SeqGAN model to synthesize cluster sequences. Using the synthesized sequences, we are able to achieve better performance on high-dimension data training and prediction compared to the state-of-the-art models.
    參考文獻: * [1]  P. S. Churchland, T. J. Sejnowski, and T. A. Poggio, The computational brain. MIT press, 2016.
    * [2]  W. Awad and S. ELseuofi, “Machine learning methods for e-mail classification,” International Journal of Computer Applications, vol. 16, no. 1, 2011.
    * [3]  F. Sebastiani, “Machine learning in automated text categorization,” ACM computing surveys (CSUR), vol. 34, no. 1, pp. 1–47, 2002.
    * [4]  W. B. Rauch-Hindin, Artificial Intelligence in Business, Science, and Industry: Fun- damentals. Prentice-Hall New Jersey, 1986.
    * [5]  K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
    * [6]  Y. LeCun, Y. Bengio et al., “Convolutional networks for images, speech, and time series,” The handbook of brain theory and neural networks, vol. 3361, no. 10, p. 1995, 1995.
    * [7]  R. J. Williams and D. Zipser, “A learning algorithm for continually running fully recurrent neural networks,” Neural computation, vol. 1, no. 2, pp. 270–280, 1989.
    * [8]  I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in neural information processing systems, 2014, pp. 2672–2680.
    * [9]  O. Abdel-Hamid, A.-r. Mohamed, H. Jiang, L. Deng, G. Penn, and D. Yu, “Convolu- tional neural networks for speech recognition,” IEEE/ACM Transactions on audio, speech, and language processing, vol. 22, no. 10, pp. 1533–1545, 2014.
    * [10]  I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” in Advances in neural information processing systems, 2014, pp. 3104– 3112.
    * [11]  A. Graves, A.-r. Mohamed, and G. Hinton, “Speech recognition with deep recur- rent neural networks,” in Acoustics, speech and signal processing (icassp), 2013 ieee international conference on. IEEE, 2013, pp. 6645–6649.
    * [12]  S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997.
    * [13]  F. A. Gers, J. Schmidhuber, and F. Cummins, Learning to forget: Continual predic- tion with LSTM. IET, 1999.
    * [14]  L. Yu, W. Zhang, J. Wang, and Y. Yu, “Seqgan: Sequence generative adversarial nets with policy gradient.” in AAAI, 2017, pp. 2852–2858.
    * [15]  A. Creswell and A. A. Bharath, “Denoising adversarial autoencoders,” IEEE trans- actions on neural networks and learning systems, no. 99, pp. 1–17, 2018.
    * [16]  A. Makhzani, J. Shlens, N. Jaitly, I. Goodfellow, and B. Frey, “Adversarial autoen- coders,” arXiv preprint arXiv:1511.05644, 2015.
    * [17]  A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learn- ing with deep convolutional generative adversarial networks,” arXiv preprint arXiv:1511.06434, 2015.
    * [18]  T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, “Im- proved techniques for training gans,” in Advances in Neural Information Processing Systems, 2016, pp. 2234–2242.
    * [19]  P. Vincent, H. Larochelle, Y. Bengio, and P.-A. Manzagol, “Extracting and compos- ing robust features with denoising autoencoders,” in Proceedings of the 25th inter- national conference on Machine learning. ACM, 2008, pp. 1096–1103.
    * [20]  P. B. Myszkowski and B. Buczek, “Growing hierarchical self-organizing map for searching documents using visual content,” in 2011 Federated Conference on Com- puter Science and Information Systems (FedCSIS). IEEE, 2011, pp. 77–81.
    * [21]  J. Dai, Y. Lu, and Y.-N. Wu, “Generative modeling of convolutional neural net- works,” arXiv preprint arXiv:1412.6296, 2014.
    * [22]  J. Xie, Y. Lu, S.-C. Zhu, and Y. Wu, “A theory of generative convnet,” in Interna- tional Conference on Machine Learning, 2016, pp. 2635–2644.
    * [23]  J. Xie, Y. Lu, R. Gao, S.-C. Zhu, and Y. N. Wu, “Cooperative training of descriptor and generator networks,” arXiv preprint arXiv:1609.09408, 2016.
    * [24]  Z. Dai, Z. Yang, F. Yang, W. W. Cohen, and R. R. Salakhutdinov, “Good semi- supervised learning that requires a bad gan,” in Advances in Neural Information Processing Systems, 2017, pp. 6510–6520.
    * [25]  E. L. Denton, S. Chintala, R. Fergus et al., “Deep generative image models using a laplacian pyramid of adversarial networks,” in Advances in neural information processing systems, 2015, pp. 1486–1494.
    * [26]  J. Ho and S. Ermon, “Generative adversarial imitation learning,” in Advances in Neural Information Processing Systems, 2016, pp. 4565–4573.
    * [27]  P. Luc, C. Couprie, S. Chintala, and J. Verbeek, “Semantic segmentation using adversarial networks,” arXiv preprint arXiv:1611.08408, 2016.
    * [28]  M.-Y. Liu and O. Tuzel, “Coupled generative adversarial networks,” in Advances in neural information processing systems, 2016, pp. 469–477.
    * [29]  J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” arXiv preprint, 2017.
    * [30]  S. Tulyakov, M.-Y. Liu, X. Yang, and J. Kautz, “Mocogan: Decomposing motion and content for video generation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1526–1535.
    * [31]  M. Saito, E. Matsumoto, and S. Saito, “Temporal generative adversarial nets with singular value clipping,” in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2830–2839.
    * [32]  C. Vondrick, H. Pirsiavash, and A. Torralba, “Generating videos with scene dynam- ics,” in Advances In Neural Information Processing Systems, 2016, pp. 613–621.
    * [33]  J. Xie, R. Gao, Z. Zheng, S.-C. Zhu, and Y. N. Wu, “Learning dynamic generator model by alternating back-propagation through time,” arXiv preprint arXiv:1812.10587, 2018.
    * [34]  A. Hadriche, N. Jmail, and R. Elleuch, “Different methods of partitioning the phase space of a dynamic system,” International Journal of Computer Applications, vol. 93, pp. 1–5, 05 2014.
    * [35]  M. Dittenbach, D. Merkl, and A. Rauber, “The growing hierarchical self-organizing map,” in Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium, vol. 6. IEEE, 2000, pp. 15–19.
    * [36]  J. Macqueen, “Some methods for classification and analysis of multivariate observa- tions,” in In 5-th Berkeley Symposium on Mathematical Statistics and Probability, 1967, pp. 281–297.
    * [37]  T. Kohonen, Neurocomputing: Foundations of Research, J. A. Anderson and E. Rosenfeld, Eds. Cambridge, MA, USA: MIT Press, 1988. [Online]. Available: http://dl.acm.org/citation.cfm?id=65669.104428
    * [38]  M. D. E. P. Andreas Rauber, Dieter Merkl, “The growing hierarchical self-organizing map.” [Online]. Available: http://www.ifs.tuwien.ac.at/∼andi/ghsom/
    * [39]  V. Rajagopalan, A. Ray, R. Samsi, and J. Mayer, “Pattern identification in dynamical systems via symbolic time series analysis,” Pattern Recognition, vol. 40, no. 11, pp. 2897–2907, 2007.
    * [40]  T. Y. Lin, H. H. C. Chuang, and F. Yu, “Tracking supply chain process variabil- ity with unsupervised cluster traversal,” in 2018 IEEE 16th Intl Conf on Depend- able, Autonomic and Secure Computing, 16th Intl Conf on Pervasive Intelligence and Computing, 4th Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech). IEEE, 2018, pp. 966–973.
    * [41]  J. R. Quinlan et al., “Bagging, boosting, and c4. 5,” in AAAI/IAAI, Vol. 1, 1996, pp. 725–730.
    * [42]  J. Demˇsar, T. Curk, A. Erjavec, Cˇrt Gorup, T. Hoˇcevar, M. Milutinoviˇc, M. Moˇzina, M. Polajnar, M. Toplak, A. Stariˇc, M. Sˇtajdohar, L. Umek, L. Zˇagar, J. Zˇbontar, M. Zˇitnik, and B. Zupan, “Orange: Data mining toolbox in python,” Journal of Machine Learning Research, vol. 14, pp. 2349–2353, 2013. [Online]. Available: http://jmlr.org/papers/v14/demsar13a.html
    描述: 碩士
    國立政治大學
    資訊管理學系
    106356004
    資料來源: http://thesis.lib.nccu.edu.tw/record/#G0106356004
    資料類型: thesis
    DOI: 10.6814/NCCU201900564
    顯示於類別:[資訊管理學系] 學位論文

    文件中的檔案:

    檔案 大小格式瀏覽次數
    600401.pdf1373KbAdobe PDF20檢視/開啟


    在政大典藏中所有的資料項目都受到原著作權保護.


    社群 sharing

    著作權政策宣告 Copyright Announcement
    1.本網站之數位內容為國立政治大學所收錄之機構典藏，無償提供學術研究與公眾教育等公益性使用，惟仍請適度，合理使用本網站之內容，以尊重著作權人之權益。商業上之利用，則請先取得著作權人之授權。
    The digital content of this website is part of National Chengchi University Institutional Repository. It provides free access to academic research and public education for non-commercial use. Please utilize it in a proper and reasonable manner and respect the rights of copyright owners. For commercial use, please obtain authorization from the copyright owner in advance.

    2.本網站之製作，已盡力防止侵害著作權人之權益，如仍發現本網站之數位內容有侵害著作權人權益情事者，請權利人通知本網站維護人員(nccur@nccu.edu.tw)，維護人員將立即採取移除該數位著作等補救措施。
    NCCU Institutional Repository is made to protect the interests of copyright owners. If you believe that any material on the website infringes copyright, please contact our staff(nccur@nccu.edu.tw). We will remove the work from the repository and investigate your claim.
    DSpace Software Copyright © 2002-2004  MIT &  Hewlett-Packard  /   Enhanced by   NTU Library IR team Copyright ©   - 回饋