Loading...
|
Please use this identifier to cite or link to this item:
https://nccur.lib.nccu.edu.tw/handle/140.119/124710
|
| Title: | 強記暨軟化整合演算法：以ReLU激發函數與二元輸入/輸出為例
The Cramming, Softening and Integrating Learning Algorithm with ReLU activation function for Binary Input/Output Problems |
| Authors: | 蔡羽涵
Tsai, Yu-Han |
| Contributors: | 蔡瑞煌
蕭舜文
Tsaih, Rua-Huan
Hsiao, Shun-Wen
蔡羽涵
Tsai, Yu-Han |
| Keywords: | 強記暨軟化整合
自適應神經網路
圖形處理單元
ReLU
TensorFlow
GPU |
| Date: | 2019 |
| Issue Date: | 2019-08-07 16:06:51 (UTC+8) |
| Abstract: | 在類神經網路領域中，很少研究會同時針對以下三個議題進行研究：
(1) 在學習過程中，神經網路能夠有系統的調整隱藏節點的數量 ;
(2) 使用ReLU作為激發函數，而非使用傳統的tanh ;
(3) 保證能學習所有的訓練資料。
在本研究中會針對上述三點，提出強記暨軟化整合 (Cramming, Softening and Integrating)學習演算法，基於單層神經網路並使用ReLU作為激發函數，解決二元輸入/輸出問題，此外也會進行實驗驗證演算法。在實驗中我們使用SPECT心臟影像資料進行實驗，並且使用張量流(TensorFlow)和圖形處理單元(GPU)進行實作。
Rare Artificial Neural Networks studies address simultaneously the challenges of (1) systematically adjusting the amount of used hidden layer nodes within the learning process, (2) adopting ReLU activation function instead of tanh function for fast learning, and (3) guaranteeing learning all training data. This study will address these challenges through deriving the CSI (Cramming, Softening and Integrating) learning algorithm for the single-hidden layer feed-forward neural networks with ReLU activation function and the binary input/output, and further making the technical justification. For the purpose of verifying the proposed learning algorithm, this study conducts an empirical experiment using SPECT heart diagnosis data set from UCI Machine Learning repository. The learning algorithm is implemented via the advanced TensorFlow and GPU. |
| Reference: | [1] I. C. Yeh, and C. H. Lien, "The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients," Expert Systems with Applications, vol. 36(2), pp. 2473-2480, 2009.
[2] J. de Jesús Rubio, E. Lughofer, J. A. Meda-Campaña, L. A. Páramo, J. F. Novoa, and J. Pacheco, “Neural network updating via argument Kalman filter for modeling of Takagi-Sugeno fuzzy models,” Journal of Intelligent & Fuzzy Systems, vol. 35(2), pp. 2585-2596, 2018.
[3] X. L. Meng, F. G. Shi, and J. C. Yao, “An inequality approach for evaluating decision making units with a fuzzy output,” Journal of Intelligent & Fuzzy Systems, vol. 34(1), pp. 459-465, 2018.
[4] J. de Jesús Rubio, “Stable Kalman filter and neural network for the chaotic systems identification,” Journal of the Franklin Institute, vol. 354(16), pp. 7444-7462, 2017.
[5] M. Y. Cheng, D. Prayogo, and Y. W. Wu, “Prediction of permanent deformation in asphalt pavements using a novel symbiotic organisms search-least squares support vector regression,” Neural Computing and Applications, 2018.
[6] J. de Jesús Rubio, “SOFMLS: online self-organizing fuzzy modified least-squares network, “ IEEE Transactions on Fuzzy Systems, vol. 17(6), pp. 1296-1309, 2009.
[7] X. M. Zhang, and Q. L. Han, “State estimation for static neural networks with time-varying delays based on an improved reciprocally convex inequality,” IEEE Transactions on Neural Networks and Learning Systems, vol. 29(4), pp. 1376-1381, 2018.
[8] V. Nair, and G. E. Hinton, “Rectified Linear Units improve restricted boltzman machines,” Proceedings of the 27th international conference on machine learning (ICML-10), pp. 807-814, 2010.
[9] L. Ma, and K. Khorasani, "A new strategy for adaptively constructing multilayer feedforward neural networks," Neurocomputing, vol. 51, pp. 361-385, 2003.
[10] R. R. Tsaih, “An explanation of reasoning neural networks,” Mathematical and Computer Modelling, vol. 28(2), pp 37-44, 1998.
[11] E. Watanabe, and H. Shimizu, “Algorithm for pruning hidden nodes in multi-layered neural network for binary pattern classification problem,” Proceeding of 1993 International Joint Conference on Neural Networks I, pp. 327-330, 1993.
[12] Y. Q. Chen, D. W. Thomas, and M. S. Nixon, "Generating-shrinking algorithm for learning arbitrary classification," Neural Networks, vol. 7(9), pp. 1477-1489, 1994.
[13] M. Mezard, and J. P. Nadal, "Learning in feedforward layered networks: The tiling algorithm," Journal of Physics A: Mathematical and General, vol. 22(12), pp. 2191, 1989.
[14] S. E. Fahlman, and C. Lebiere, "The cascade-correlation learning architecture," Advances in neural information processing systems, pp. 524-532, 1990.
[15] M. Frean, "The upstart algorithm: A method for constructing and training feedforward neural networks," Neural computation, vol. 2(2), pp. 198-209, 1990.
[16] R. R. Tsaih, "The softening learning procedure," Mathematical and computer modelling, vol. 18(8), pp. 61-64, 1993.
[17] R. H. Tsaih, and T. C. Cheng, “A resistant learning procedure for coping with outliers,” Annals of Mathematics and Artificial Intelligence, vol. 57(2), pp. 161-180, 2009.
[18] R. H. Tsaih, B. S. Kuo, T. H. Lin, and C. C. Hsu, “The use of big data analytics to predict the foreign exchange rate based on public media: A machine-learning experiment,” IT Professional, vol. 20(2), pp. 34-41, 2018.
[19] L. A. Kurgan, K. J. Cios, R. Tadeusiewicz, M. Ogiela, and L. S. Goodenday, "Knowledge Discovery Approach to Automated Cardiac SPECT Diagnosis," Artificial Intelligence in Medicine, vol. 23(2), pp. 149-169, Oct 2001.
[20] D. Dua, and E. Karra Taniskidou, (2017). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.
[21] Y. LeCun, Y. Bengio, and G. Hinton, “Deep Learning,” nature, vol. 521(7553), pp. 436, 2015.
[22] K. Hara, D. Saito, and H. Shouno, “Analysis of function of rectified linear unit used in deep learning,” 2015 International Joint Conference on Neural Networks (IJCNN). IEEE, pp. 1-8, 2015.
[23] B. Xu, N. Wang, T. Chen, and M. Li, “Empirical evaluation of rectified activations in convolutional network,” arXiv preprint, arXiv:1505.00853, 2015.
[24] S. Y. Huang, J. W. Lin, and R. H. Tsaih, “Outlier detection in the concept drifting environment,” IEEE 2016 International Joint Conference Neural Networks, pp.31-37, 2016.
[25] M. Abadi, P. Barham, et al., “Tensorflow: A system for large-scale machine learning,” In 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16), pp.265-283, 2016.
[26] M. Abadi, A. Agarwal et al., “TensorFlow: Large-scale machine learning on heterogeneous distributed systems,” arXiv preprint, arXiv:1603.04467, 2016.
[27] S. Ruder, “An overview of gradient descent optimization algorithms,” arXiv preprint, arXiv:1609.04747.
[28] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic Minority Over-sampling Technique,” Journal of artificial intelligence research, vol. 16, pp. 321-357, 2002.
[29] X. Y. Liu, J. Wu, and Z. H. Zhou, “Exploratory undersampling for class-imbalance learning,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 39(2), pp. 539-550. |
| Description: | 碩士
國立政治大學
資訊管理學系
106356018 |
| Source URI: | http://thesis.lib.nccu.edu.tw/record/#G0106356018 |
| Data Type: | thesis |
| DOI: | 10.6814/NCCU201900582 |
| Appears in Collections: | [資訊管理學系] 學位論文
|
Files in This Item:
| File |
Size | Format | |
|---|
| 601801.pdf | 1205Kb | Adobe PDF2 | 0 | View/Open |
|
All items in 政大典藏 are protected by copyright, with all rights reserved.
|
