政大機構典藏-National Chengchi University Institutional Repository(NCCUR):Item 140.119/149072

政大典藏 > College of Informatics > Executive Master Program of Computer Science of NCCU > Theses > Item 140.119/149072

Please use this identifier to cite or link to this item: https://nccur.lib.nccu.edu.tw/handle/140.119/149072

Title:	基於多輸出標籤概念之階層式深度神經網路架構 Hierarchical Deep Neural Network Architecture based on Multi-Output Concept
Authors:	索立桐 So, Li-Tung
Contributors:	廖文宏 Liao, Wen-Hung 索立桐 So, Li-Tung
Keywords:	深度學習多輸出分類器階層式分類階層一致性預測風險 Deep learning multi-output classifier hierarchical classification hierarchical consistency prediction risk
Date:	2023
Issue Date:	2024-01-02 15:39:30 (UTC+8)
Abstract:	階層式深度神經網路是一種能強化現有直觀平坦分類的深度學習框架，有效確保模型在學習的路線上路徑正確（由粗分類至細分類），並能在標註資料有限的情況下，針對同一目標拆分階層後，同時進行各面向的解析，使得預測出的結果可信度更高。在一般的分類任務中，資料直接在某一種共通的條件下被定義出這筆資料的類別並賦予標籤，並沒有所謂的階層概念，造成容易區分的資料類型和難以區分的資料類型混雜在一起，導致後續的分類訓練上僅依靠訓練模型本身的擬合能力，如對資料進行階層區分後訓練，應能提升訓練的品質。本論文旨在探討於圖像分類任務中使用階層式深度學習方式，展示其能帶來的各種訓練效益，我們提出使用多輸出概念，將多階層的分類問題轉化成為一般平坦分類器可容易修改處理的形式，並加上階層一致性、預測風險等限制，期使直接而有效地套用至現有的各類分類訓練模型，調整並觀察各階層回傳的損失梯度權重，並與一般的分類方法相互比較，最後研究結果指出，使用階層式深度學習方式不僅最終的準確率不低於原始直接分類的方式，而且在我們所定義的各類指標上都得出較一般方式更佳的結果。 Hierarchical Deep Neural Networks are a type of deep learning framework that enhances existing intuitive flat classifications. They effectively ensure that the model follows the correct learning path, i.e., coarse-to-fine classification. Even with limited labeled data, they analyze various aspects simultaneously by splitting the training sets into sub-classes according to a predefined hierarchy. This approach increases the credibility of the predicted results. In typical classification tasks, data is directly assigned to a category and labeled based on a common set of conditions, without the concept of hierarchy. This leads to a mixture of easily distinguishable and difficult-to-distinguish data types, relying solely on the fitting capability of the training model in subsequent classification training. Therefore, training data with hierarchical divisions should improve the quality of training. This thesis aims to explore the use of hierarchical deep learning in image classification tasks and demonstrate the various training benefits it brings. We propose the utilization of the multi-label concept to convert multi-level classification problems into a form that can be easily modified and processed by general flat classifiers. We also incorporate constraints such as hierarchical consistency and prediction risk to enable direct and effective application to various existing classification training models. During the process, we adjust and observe the loss gradients weights returned by each hierarchy level, and compare them with conventional classification methods. The research results indicate that using the hierarchical deep learning approach not only achieves accuracy comparable to or higher than the original direct classification method but also yields better results in various performance indicators.
Reference:	[1] 朱家宏. 階層式深度神經網路及其應用, pages 18-20, 2023. [2] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, 2016. [3] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., ... & Rabinovich, A. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1-9), 2015. [4] Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, Hartwig Adam. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv 2017, arXiv: 1704.04861.2017. [5] Dmitry Retinskiy. (2020). Multi-Label Image Classification with PyTorch. Retrieved from https://learnopencv.com/multi-label-image-classification-with-pytorch(Oct.13,2023) [6] CNN for deep learning \| Convolutional neural networks. Retrieved from https://datapeaker.com/en/big--data/cnn-for-deep-learning-convolutional-neural-networks(Oct.13,2023) [7] LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324. [8] Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90. [9] K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv: 1409.1556., 2014. [10] Yan, Z., Zhang, H., Piramuthu, R., Jagadeesh, V., DeCoste, D., Di, W., & Yu, Y.. HD-CNN: hierarchical deep convolutional neural networks for large scale visual recognition. In Proceedings of the IEEE international conference on computer vision (pp. 2740-2748). 2015. [11] Xinqi Zhu, Michael Bain. B-CNN: Branch Convolutional Neural Network for Hierarchical Classification. arXiv 2017, arXiv:1709.09890.2017. [12] Salma Taoufiq , Balázs Nagy , Csaba Benedek.HierarchyNet: Hierarchical CNN-Based Urban Building Classification. Remote Sensing,Volume 12 ,Issue 22 .2020. [13] A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Computer Science Department, University of Toronto, Tech. Rep, 2009. [14] D. Arthur and S. Vassilvitskii. K-means++: The advantages of careful seeding. In SODA, pages 1027–1035, 2007.
Description:	碩士國立政治大學資訊科學系碩士在職專班 107971024
Source URI:	http://thesis.lib.nccu.edu.tw/record/#G0107971024
Data Type:	thesis
Appears in Collections:	[Executive Master Program of Computer Science of NCCU] Theses

Files in This Item:

File	Description	Size	Format
102401.pdf		3672Kb	Adobe PDF	7	View/Open

社群 sharing

Loading...