政大機構典藏-National Chengchi University Institutional Repository(NCCUR):Item 140.119/124710

政大機構典藏-National Chengchi University Institutional Repository(NCCUR):Item 140.119/124710

English | 正體中文 | 简体中文 | Post-Print筆數 : 27 | Items with full text/Total items : 113148/144119 (79%)
Visitors : 50711558 Online Users : 301

RC Version 6.0 © Powered By DSPACE, MIT. Enhanced by NTU Library IR team.

Scope

please add "double quotation mark" for query phrases to get precise results

please goto advance search for comprehansive author search

Adv. Search

Home ‧ Login ‧ Upload ‧ Help ‧ About ‧ Administer

Goto mobile version

政大典藏 > College of Commerce > Department of MIS > Theses > Item 140.119/124710

Please use this identifier to cite or link to this item: https://nccur.lib.nccu.edu.tw/handle/140.119/124710

Title:	強記暨軟化整合演算法：以ReLU激發函數與二元輸入/輸出為例 The Cramming, Softening and Integrating Learning Algorithm with ReLU activation function for Binary Input/Output Problems
Authors:	蔡羽涵 Tsai, Yu-Han
Contributors:	蔡瑞煌蕭舜文 Tsaih, Rua-Huan Hsiao, Shun-Wen 蔡羽涵 Tsai, Yu-Han
Keywords:	強記暨軟化整合自適應神經網路圖形處理單元 ReLU TensorFlow GPU
Date:	2019
Issue Date:	2019-08-07 16:06:51 (UTC+8)
Abstract:	在類神經網路領域中，很少研究會同時針對以下三個議題進行研究： (1) 在學習過程中，神經網路能夠有系統的調整隱藏節點的數量 ; (2) 使用ReLU作為激發函數，而非使用傳統的tanh ; (3) 保證能學習所有的訓練資料。在本研究中會針對上述三點，提出強記暨軟化整合 (Cramming, Softening and Integrating)學習演算法，基於單層神經網路並使用ReLU作為激發函數，解決二元輸入/輸出問題，此外也會進行實驗驗證演算法。在實驗中我們使用SPECT心臟影像資料進行實驗，並且使用張量流(TensorFlow)和圖形處理單元(GPU)進行實作。 Rare Artificial Neural Networks studies address simultaneously the challenges of (1) systematically adjusting the amount of used hidden layer nodes within the learning process, (2) adopting ReLU activation function instead of tanh function for fast learning, and (3) guaranteeing learning all training data. This study will address these challenges through deriving the CSI (Cramming, Softening and Integrating) learning algorithm for the single-hidden layer feed-forward neural networks with ReLU activation function and the binary input/output, and further making the technical justification. For the purpose of verifying the proposed learning algorithm, this study conducts an empirical experiment using SPECT heart diagnosis data set from UCI Machine Learning repository. The learning algorithm is implemented via the advanced TensorFlow and GPU.
Reference:	[1] I. C. Yeh, and C. H. Lien, "The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients," Expert Systems with Applications, vol. 36(2), pp. 2473-2480, 2009. [2] J. de Jesús Rubio, E. Lughofer, J. A. Meda-Campaña, L. A. Páramo, J. F. Novoa, and J. Pacheco, “Neural network updating via argument Kalman filter for modeling of Takagi-Sugeno fuzzy models,” Journal of Intelligent & Fuzzy Systems, vol. 35(2), pp. 2585-2596, 2018. [3] X. L. Meng, F. G. Shi, and J. C. Yao, “An inequality approach for evaluating decision making units with a fuzzy output,” Journal of Intelligent & Fuzzy Systems, vol. 34(1), pp. 459-465, 2018. [4] J. de Jesús Rubio, “Stable Kalman filter and neural network for the chaotic systems identification,” Journal of the Franklin Institute, vol. 354(16), pp. 7444-7462, 2017. [5] M. Y. Cheng, D. Prayogo, and Y. W. Wu, “Prediction of permanent deformation in asphalt pavements using a novel symbiotic organisms search-least squares support vector regression,” Neural Computing and Applications, 2018. [6] J. de Jesús Rubio, “SOFMLS: online self-organizing fuzzy modified least-squares network, “ IEEE Transactions on Fuzzy Systems, vol. 17(6), pp. 1296-1309, 2009. [7] X. M. Zhang, and Q. L. Han, “State estimation for static neural networks with time-varying delays based on an improved reciprocally convex inequality,” IEEE Transactions on Neural Networks and Learning Systems, vol. 29(4), pp. 1376-1381, 2018. [8] V. Nair, and G. E. Hinton, “Rectified Linear Units improve restricted boltzman machines,” Proceedings of the 27th international conference on machine learning (ICML-10), pp. 807-814, 2010. [9] L. Ma, and K. Khorasani, "A new strategy for adaptively constructing multilayer feedforward neural networks," Neurocomputing, vol. 51, pp. 361-385, 2003. [10] R. R. Tsaih, “An explanation of reasoning neural networks,” Mathematical and Computer Modelling, vol. 28(2), pp 37-44, 1998. [11] E. Watanabe, and H. Shimizu, “Algorithm for pruning hidden nodes in multi-layered neural network for binary pattern classification problem,” Proceeding of 1993 International Joint Conference on Neural Networks I, pp. 327-330, 1993. [12] Y. Q. Chen, D. W. Thomas, and M. S. Nixon, "Generating-shrinking algorithm for learning arbitrary classification," Neural Networks, vol. 7(9), pp. 1477-1489, 1994. [13] M. Mezard, and J. P. Nadal, "Learning in feedforward layered networks: The tiling algorithm," Journal of Physics A: Mathematical and General, vol. 22(12), pp. 2191, 1989. [14] S. E. Fahlman, and C. Lebiere, "The cascade-correlation learning architecture," Advances in neural information processing systems, pp. 524-532, 1990. [15] M. Frean, "The upstart algorithm: A method for constructing and training feedforward neural networks," Neural computation, vol. 2(2), pp. 198-209, 1990. [16] R. R. Tsaih, "The softening learning procedure," Mathematical and computer modelling, vol. 18(8), pp. 61-64, 1993. [17] R. H. Tsaih, and T. C. Cheng, “A resistant learning procedure for coping with outliers,” Annals of Mathematics and Artificial Intelligence, vol. 57(2), pp. 161-180, 2009. [18] R. H. Tsaih, B. S. Kuo, T. H. Lin, and C. C. Hsu, “The use of big data analytics to predict the foreign exchange rate based on public media: A machine-learning experiment,” IT Professional, vol. 20(2), pp. 34-41, 2018. [19] L. A. Kurgan, K. J. Cios, R. Tadeusiewicz, M. Ogiela, and L. S. Goodenday, "Knowledge Discovery Approach to Automated Cardiac SPECT Diagnosis," Artificial Intelligence in Medicine, vol. 23(2), pp. 149-169, Oct 2001. [20] D. Dua, and E. Karra Taniskidou, (2017). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science. [21] Y. LeCun, Y. Bengio, and G. Hinton, “Deep Learning,” nature, vol. 521(7553), pp. 436, 2015. [22] K. Hara, D. Saito, and H. Shouno, “Analysis of function of rectified linear unit used in deep learning,” 2015 International Joint Conference on Neural Networks (IJCNN). IEEE, pp. 1-8, 2015. [23] B. Xu, N. Wang, T. Chen, and M. Li, “Empirical evaluation of rectified activations in convolutional network,” arXiv preprint, arXiv:1505.00853, 2015. [24] S. Y. Huang, J. W. Lin, and R. H. Tsaih, “Outlier detection in the concept drifting environment,” IEEE 2016 International Joint Conference Neural Networks, pp.31-37, 2016. [25] M. Abadi, P. Barham, et al., “Tensorflow: A system for large-scale machine learning,” In 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16), pp.265-283, 2016. [26] M. Abadi, A. Agarwal et al., “TensorFlow: Large-scale machine learning on heterogeneous distributed systems,” arXiv preprint, arXiv:1603.04467, 2016. [27] S. Ruder, “An overview of gradient descent optimization algorithms,” arXiv preprint, arXiv:1609.04747. [28] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic Minority Over-sampling Technique,” Journal of artificial intelligence research, vol. 16, pp. 321-357, 2002. [29] X. Y. Liu, J. Wu, and Z. H. Zhou, “Exploratory undersampling for class-imbalance learning,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 39(2), pp. 539-550.
Description:	碩士國立政治大學資訊管理學系 106356018
Source URI:	http://thesis.lib.nccu.edu.tw/record/#G0106356018
Data Type:	thesis
DOI:	10.6814/NCCU201900582
Appears in Collections:	[Department of MIS] Theses

Files in This Item:

File	Size	Format
601801.pdf	1205Kb	Adobe PDF2	0	View/Open

All items in 政大典藏 are protected by copyright, with all rights reserved.

社群 sharing

著作權政策宣告 Copyright Announcement

1.本網站之數位內容為國立政治大學所收錄之機構典藏，無償提供學術研究與公眾教育等公益性使用，惟仍請適度，合理使用本網站之內容，以尊重著作權人之權益。商業上之利用，則請先取得著作權人之授權。
The digital content of this website is part of National Chengchi University Institutional Repository. It provides free access to academic research and public education for non-commercial use. Please utilize it in a proper and reasonable manner and respect the rights of copyright owners. For commercial use, please obtain authorization from the copyright owner in advance.

2.本網站之製作，已盡力防止侵害著作權人之權益，如仍發現本網站之數位內容有侵害著作權人權益情事者，請權利人通知本網站維護人員(nccur@nccu.edu.tw)，維護人員將立即採取移除該數位著作等補救措施。
NCCU Institutional Repository is made to protect the interests of copyright owners. If you believe that any material on the website infringes copyright, please contact our staff(nccur@nccu.edu.tw). We will remove the work from the repository and investigate your claim.

DSpace Software Copyright © 2002-2004 MIT & Hewlett-Packard / Enhanced by NTU Library IR team Copyright © - Feedback