Loading...
|
Please use this identifier to cite or link to this item:
https://nccur.lib.nccu.edu.tw/handle/140.119/124705
|
| Title: | HiSeqGAN: 高維資料的序列合成與預測
HiSeqGAN: High-dimensional Sequence Synthesis and Prediction |
| Authors: | 田韻杰
Tien, Yun-Chieh |
| Contributors: | 郁方
Yu, Fang
田韻杰
Tien, Yun-Chieh |
| Keywords: | 高維資料
序列合成
序列預測
增長層級式自我組織映射圖
序列對抗生成網路
High-dimensional data
Sequence Synthesis
Sequence Prediction
SeqGAN
GHSOM |
| Date: | 2019 |
| Issue Date: | 2019-08-07 16:05:57 (UTC+8) |
| Abstract: | 隨著大數據時代來臨，許多資料都具有高維度的變數，而要如何提升高維度序列資料的預測準確度是重要的課題之一。本篇論文即是結合了深度學習的技術，針對高維度資料的序列提出一個有效合成和預測的新方法：HiSeqGAN。
首先，我們會利用增長層級式自我組織映射圖(GHSOM)為資料進行結構化分群，接著透過編碼演算法將分群後的資料轉換成座標向量，給予高維度資料一個新的編碼方式。並採用序列對抗生成網路(SeqGAN)作為主要的訓練模型，以此合成和預測結構化資料的序列。
本篇論文的貢獻在於利用序列對抗生成網路對結構化資料進行合成及預測，除了可以提供更多的資料用來訓練一個性能較好的遞歸神經網路 (RNN)模型之外，也能有效的提升預測結構化資料的準確性。
High-dimensional data sequences constantly appear in practice. State-of-the-art models such as recurrent neural networks suffer prediction accuracy from complex relations among values of attributes. Adopting unsupervised clustering that clusters data based on their attribute value similarity results data in lower dimensions that can be structured in a hierarchical relation. It is essential to consider these data relations to improve the performance of training models. In this work, we propose a new approach to synthesize and predict sequences of data that are structured in a hierarchy. Specifically, we adopt a new hierarchical data encoding and seamlessly modify loss functions of SeqGAN as our training model to synthesize data sequences. In practice, we first use the hierarchical clustering algorithm, GHSOM, to cluster our training data. By relabelling a sample with the cluster that it falls to, we are able to use the GHSOM map to identify the hierarchical relation of samples. We then converse the clusters to the coordinate vectors with our hierarchical data encoding algorithm and replace the loss function with maximizing cosine similarity in the SeqGAN model to synthesize cluster sequences. Using the synthesized sequences, we are able to achieve better performance on high-dimension data training and prediction compared to the state-of-the-art models. |
| Reference: | * [1] P. S. Churchland, T. J. Sejnowski, and T. A. Poggio, The computational brain. MIT press, 2016.
* [2] W. Awad and S. ELseuofi, “Machine learning methods for e-mail classification,” International Journal of Computer Applications, vol. 16, no. 1, 2011.
* [3] F. Sebastiani, “Machine learning in automated text categorization,” ACM computing surveys (CSUR), vol. 34, no. 1, pp. 1–47, 2002.
* [4] W. B. Rauch-Hindin, Artificial Intelligence in Business, Science, and Industry: Fun- damentals. Prentice-Hall New Jersey, 1986.
* [5] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
* [6] Y. LeCun, Y. Bengio et al., “Convolutional networks for images, speech, and time series,” The handbook of brain theory and neural networks, vol. 3361, no. 10, p. 1995, 1995.
* [7] R. J. Williams and D. Zipser, “A learning algorithm for continually running fully recurrent neural networks,” Neural computation, vol. 1, no. 2, pp. 270–280, 1989.
* [8] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in neural information processing systems, 2014, pp. 2672–2680.
* [9] O. Abdel-Hamid, A.-r. Mohamed, H. Jiang, L. Deng, G. Penn, and D. Yu, “Convolu- tional neural networks for speech recognition,” IEEE/ACM Transactions on audio, speech, and language processing, vol. 22, no. 10, pp. 1533–1545, 2014.
* [10] I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” in Advances in neural information processing systems, 2014, pp. 3104– 3112.
* [11] A. Graves, A.-r. Mohamed, and G. Hinton, “Speech recognition with deep recur- rent neural networks,” in Acoustics, speech and signal processing (icassp), 2013 ieee international conference on. IEEE, 2013, pp. 6645–6649.
* [12] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997.
* [13] F. A. Gers, J. Schmidhuber, and F. Cummins, Learning to forget: Continual predic- tion with LSTM. IET, 1999.
* [14] L. Yu, W. Zhang, J. Wang, and Y. Yu, “Seqgan: Sequence generative adversarial nets with policy gradient.” in AAAI, 2017, pp. 2852–2858.
* [15] A. Creswell and A. A. Bharath, “Denoising adversarial autoencoders,” IEEE trans- actions on neural networks and learning systems, no. 99, pp. 1–17, 2018.
* [16] A. Makhzani, J. Shlens, N. Jaitly, I. Goodfellow, and B. Frey, “Adversarial autoen- coders,” arXiv preprint arXiv:1511.05644, 2015.
* [17] A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learn- ing with deep convolutional generative adversarial networks,” arXiv preprint arXiv:1511.06434, 2015.
* [18] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, “Im- proved techniques for training gans,” in Advances in Neural Information Processing Systems, 2016, pp. 2234–2242.
* [19] P. Vincent, H. Larochelle, Y. Bengio, and P.-A. Manzagol, “Extracting and compos- ing robust features with denoising autoencoders,” in Proceedings of the 25th inter- national conference on Machine learning. ACM, 2008, pp. 1096–1103.
* [20] P. B. Myszkowski and B. Buczek, “Growing hierarchical self-organizing map for searching documents using visual content,” in 2011 Federated Conference on Com- puter Science and Information Systems (FedCSIS). IEEE, 2011, pp. 77–81.
* [21] J. Dai, Y. Lu, and Y.-N. Wu, “Generative modeling of convolutional neural net- works,” arXiv preprint arXiv:1412.6296, 2014.
* [22] J. Xie, Y. Lu, S.-C. Zhu, and Y. Wu, “A theory of generative convnet,” in Interna- tional Conference on Machine Learning, 2016, pp. 2635–2644.
* [23] J. Xie, Y. Lu, R. Gao, S.-C. Zhu, and Y. N. Wu, “Cooperative training of descriptor and generator networks,” arXiv preprint arXiv:1609.09408, 2016.
* [24] Z. Dai, Z. Yang, F. Yang, W. W. Cohen, and R. R. Salakhutdinov, “Good semi- supervised learning that requires a bad gan,” in Advances in Neural Information Processing Systems, 2017, pp. 6510–6520.
* [25] E. L. Denton, S. Chintala, R. Fergus et al., “Deep generative image models using a laplacian pyramid of adversarial networks,” in Advances in neural information processing systems, 2015, pp. 1486–1494.
* [26] J. Ho and S. Ermon, “Generative adversarial imitation learning,” in Advances in Neural Information Processing Systems, 2016, pp. 4565–4573.
* [27] P. Luc, C. Couprie, S. Chintala, and J. Verbeek, “Semantic segmentation using adversarial networks,” arXiv preprint arXiv:1611.08408, 2016.
* [28] M.-Y. Liu and O. Tuzel, “Coupled generative adversarial networks,” in Advances in neural information processing systems, 2016, pp. 469–477.
* [29] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” arXiv preprint, 2017.
* [30] S. Tulyakov, M.-Y. Liu, X. Yang, and J. Kautz, “Mocogan: Decomposing motion and content for video generation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1526–1535.
* [31] M. Saito, E. Matsumoto, and S. Saito, “Temporal generative adversarial nets with singular value clipping,” in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2830–2839.
* [32] C. Vondrick, H. Pirsiavash, and A. Torralba, “Generating videos with scene dynam- ics,” in Advances In Neural Information Processing Systems, 2016, pp. 613–621.
* [33] J. Xie, R. Gao, Z. Zheng, S.-C. Zhu, and Y. N. Wu, “Learning dynamic generator model by alternating back-propagation through time,” arXiv preprint arXiv:1812.10587, 2018.
* [34] A. Hadriche, N. Jmail, and R. Elleuch, “Different methods of partitioning the phase space of a dynamic system,” International Journal of Computer Applications, vol. 93, pp. 1–5, 05 2014.
* [35] M. Dittenbach, D. Merkl, and A. Rauber, “The growing hierarchical self-organizing map,” in Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium, vol. 6. IEEE, 2000, pp. 15–19.
* [36] J. Macqueen, “Some methods for classification and analysis of multivariate observa- tions,” in In 5-th Berkeley Symposium on Mathematical Statistics and Probability, 1967, pp. 281–297.
* [37] T. Kohonen, Neurocomputing: Foundations of Research, J. A. Anderson and E. Rosenfeld, Eds. Cambridge, MA, USA: MIT Press, 1988. [Online]. Available: http://dl.acm.org/citation.cfm?id=65669.104428
* [38] M. D. E. P. Andreas Rauber, Dieter Merkl, “The growing hierarchical self-organizing map.” [Online]. Available: http://www.ifs.tuwien.ac.at/∼andi/ghsom/
* [39] V. Rajagopalan, A. Ray, R. Samsi, and J. Mayer, “Pattern identification in dynamical systems via symbolic time series analysis,” Pattern Recognition, vol. 40, no. 11, pp. 2897–2907, 2007.
* [40] T. Y. Lin, H. H. C. Chuang, and F. Yu, “Tracking supply chain process variabil- ity with unsupervised cluster traversal,” in 2018 IEEE 16th Intl Conf on Depend- able, Autonomic and Secure Computing, 16th Intl Conf on Pervasive Intelligence and Computing, 4th Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech). IEEE, 2018, pp. 966–973.
* [41] J. R. Quinlan et al., “Bagging, boosting, and c4. 5,” in AAAI/IAAI, Vol. 1, 1996, pp. 725–730.
* [42] J. Demˇsar, T. Curk, A. Erjavec, Cˇrt Gorup, T. Hoˇcevar, M. Milutinoviˇc, M. Moˇzina, M. Polajnar, M. Toplak, A. Stariˇc, M. Sˇtajdohar, L. Umek, L. Zˇagar, J. Zˇbontar, M. Zˇitnik, and B. Zupan, “Orange: Data mining toolbox in python,” Journal of Machine Learning Research, vol. 14, pp. 2349–2353, 2013. [Online]. Available: http://jmlr.org/papers/v14/demsar13a.html |
| Description: | 碩士
國立政治大學
資訊管理學系
106356004 |
| Source URI: | http://thesis.lib.nccu.edu.tw/record/#G0106356004 |
| Data Type: | thesis |
| DOI: | 10.6814/NCCU201900564 |
| Appears in Collections: | [資訊管理學系] 學位論文
|
Files in This Item:
| File |
Size | Format | |
|---|
| 600401.pdf | 1373Kb | Adobe PDF2 | 0 | View/Open |
|
All items in 政大典藏 are protected by copyright, with all rights reserved.
|
