English  |  正體中文  |  简体中文  |  Post-Print筆數 : 27 |  Items with full text/Total items : 113648/144635 (79%)
Visitors : 51681436      Online Users : 634
RC Version 6.0 © Powered By DSPACE, MIT. Enhanced by NTU Library IR team.
Scope Tips:
  • please add "double quotation mark" for query phrases to get precise results
  • please goto advance search for comprehansive author search
  • Adv. Search
    HomeLoginUploadHelpAboutAdminister Goto mobile version
    政大機構典藏 > 商學院 > 資訊管理學系 > 學位論文 >  Item 140.119/146895
    Please use this identifier to cite or link to this item: https://nccur.lib.nccu.edu.tw/handle/140.119/146895


    Title: 神經網路的可控制穩健性訓練研究
    Controllable Robustness Training
    Authors: 胡育騏
    Hu, Yu-Chi
    Contributors: 郁方
    Yu, Fang
    胡育騏
    Hu, Yu-Chi
    Keywords: 抽象解釋
    對抗訓練
    邏輯規則
    模型穩健性
    Abstract interpretation
    Adversarial training
    Logic rule
    Robustness
    Date: 2023
    Issue Date: 2023-09-01 14:55:10 (UTC+8)
    Abstract: 在大數據時代,神經網路技術取得了突破性進展。然而,神經網路的預測準確性和面對外界擾動和攻擊的穩健性成為一個重要的問題。使用對抗樣本或抽象解釋來訓練神經網路提高模型的穩健性可能會導致模型對原始任務的準確性和訓練性能降低。為了在準確性和穩健性之間達到平衡,我們提出了一種可控制的穩健性訓練方法,通過在對抗訓練過程中引入規則來控制神經網路模型。我們將對抗訓練的損失視為對規則的損失,從而將穩健性訓練與原始任務的訓練過程分離。在測試時,透過調整規則的強度,可以平衡模型在學習規則和約束方面的準確性和穩健性。我們證明了控制穩健性訓練的貢獻,可以在神經網路的準確性和穩健性之間達到更好的平衡。
    Neural network techniques allow for the developing of complex systems that are difficult for humans to implement. However, training these networks for robustness using adversarial examples or abstraction interpretation can reduce precision and training performance on the original task prediction. To balance the trade-off between accuracy and robustness, we propose controllable robustness training, where we control neural network models with rule representations in the adversarial training process. The loss on adversarial training can then be considered a loss on the rule, thus separating the robustness training from the original task process. Rule strength can be adjusted at a testing time on its loss ratio, which balances precision and robustness in how the model learns rules and constraints. We demonstrate that controlling the contribution of robustness training achieves a better balance of good performance in both the accuracy and robustness of neural networks.
    Reference: [1] H.-J. Yoo, “Deep convolution neural networks in computer vision: a review,” IEIE Transactions on Smart Processing and Computing, vol. 4, no. 1, pp. 35–43, 2015.
    [2] L. Deng, G. Hinton, and B. Kingsbury, “New types of deep neural network learning for speech recognition and related applications: An overview,” in 2013 IEEE international conference on acoustics, speech and signal processing. IEEE, 2013, pp.8599–8603.
    [3] J. Zhang, C. Zong, et al., “Deep neural networks in machine translation: An overview.” IEEE Intell. Syst., vol. 30, no. 5, pp. 16–25, 2015.
    [4] A. Braylan, M. Hollenbeck, E. Meyerson, and R. Miikkulainen, “Reuse of neural modules for general video game playing,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30, no. 1, 2016.
    [5] W. Zhou, J. S. Berrio, S. Worrall, and E. Nebot, “Automated evaluation of semantic segmentation robustness for autonomous driving,” IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 5, pp. 1951–1963, 2019.
    [6] F. Tramer and D. Boneh, “Adversarial training and robustness for multiple perturbations,” Advances in neural information processing systems, vol. 32, 2019.
    [7] S. Zheng, Y. Song, T. Leung, and I. Goodfellow, “Improving the robustness of deep neural networks via stability training,” in Proceedings of the ieee conference on computer vision and pattern recognition, 2016, pp. 4480–4488.
    [8] D. Zügner and S. Günnemann, “Certifiable robustness and robust training for graph convolutional networks,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019, pp. 246–256.
    [9] C. Xie, M. Tan, B. Gong, J. Wang, A. L. Yuille, and Q. V. Le, “Adversarial examples improve image recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 819–828.
    [10] A. Aldahdooh, W. Hamidouche, S. A. Fezza, and O. Déforges, “Adversarial example detection for dnn models: A review and experimental comparison,” Artificial Intelligence Review, pp. 1–60, 2022.
    [11] A. Chakraborty, M. Alam, V. Dey, A. Chattopadhyay, and D. Mukhopadhyay, “Adversarial attacks and defences: A survey,” arXiv preprint arXiv:1810.00069, 2018.
    [12] K. Han, B. Xia, and Y. Li, “2: adversarial domain adaptation to defense with adversarial perturbation removal,” Pattern Recognition, vol. 122, p. 108303, 2022.
    [13] S. Abramsky and C. Hankin, “An introduction to abstract interpretation,” in Abstract Interpretation of declarative languages, vol. 1. Ellis Horwood London, 1987, pp.63–102.
    [14] K. Ghorbal, E. Goubault, and S. Putot, “The zonotope abstract domain taylor1+,” in International conference on computer aided verification. Springer, 2009, pp.627–633.
    [15] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
    [16] S. Seo, S. Arik, J. Yoon, X. Zhang, K. Sohn, and T. Pfister, “Controlling neural networks with rule representations,” Advances in Neural Information Processing Systems, vol. 34, pp. 11 196–11 207, 2021.
    [17] B. Zhang, T. Cai, Z. Lu, D. He, and L. Wang, “Towards certifying l-infinity robustness using neural networks with l-inf-dist neurons,” in International Conference on Machine Learning. PMLR, 2021, pp. 12 368–12 379.
    [18] K. Pei, Y. Cao, J. Yang, and S. Jana, “Deepxplore: Automated whitebox testing of deep learning systems,” in proceedings of the 26th Symposium on Operating Systems
    Principles, 2017, pp. 1–18.
    [19] T. Asano, S. Bitou, M. Motoki, and N. Usui, “In-place algorithm for image rotation,” in International Symposium on Algorithms and Computation. Springer, 2007, pp.704–715.
    [20] A. Chakraborty, M. Alam, V. Dey, A. Chattopadhyay, and D. Mukhopadhyay, “A survey on adversarial attacks and defences,” CAAI Transactions on Intelligence Technology, vol. 6, no. 1, pp. 25–45, 2021.
    [21] P. Cousot and R. Cousot, “Abstract interpretation frameworks,” Journal of logic and computation, vol. 2, no. 4, pp. 511–547, 1992.
    [22] Y. Dong, F. Liao, T. Pang, H. Su, J. Zhu, X. Hu, and J. Li, “Boosting adversarial attacks with momentum,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 9185–9193.
    [23] S. Park and J. So, “On the effectiveness of adversarial training in defending against adversarial example attacks for image classification,” Applied Sciences, vol. 10, no. 22, p. 8079, 2020.
    [24] K. Ren, T. Zheng, Z. Qin, and X. Liu, “Adversarial attacks and defenses in deep learning,” Engineering, vol. 6, no. 3, pp. 346–360, 2020.
    [25] C. Si, Z. Zhang, F. Qi, Z. Liu, Y. Wang, Q. Liu, and M. Sun, “Better robustness by more coverage: Adversarial and mixup data augmentation for robust finetuning,” in Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, 2021, pp. 1569–1576.
    [26] V. Fischer, M. C. Kumar, J. H. Metzen, and T. Brox, “Adversarial examples for semantic image segmentation,” arXiv preprint arXiv:1703.01101, 2017.
    [27] H. Kwon and S. Lee, “Textual adversarial training of machine learning model for resistance to adversarial examples,” Security and Communication Networks, vol. 2022, 2022.
    [28] H. Xu, Y. Ma, H.-C. Liu, D. Deb, H. Liu, J.-L. Tang, and A. K. Jain, “Adversarial attacks and defenses in images, graphs and text: A review,” International Journal of Automation and Computing, vol. 17, no. 2, pp. 151–178, 2020.
    [29] W. Xu, D. Evans, and Y. Qi, “Feature squeezing: Detecting adversarial examples in deep neural networks,” arXiv preprint arXiv:1704.01155, 2017.
    [30] T. Bai, J. Luo, J. Zhao, B. Wen, and Q. Wang, “Recent advances in adversarial training for adversarial robustness,” arXiv preprint arXiv:2102.01356, 2021.
    [31] N. Papernot, P. McDaniel, X. Wu, S. Jha, and A. Swami, “Distillation as a defense to adversarial perturbations against deep neural networks,” in 2016 IEEE symposium on security and privacy (SP). IEEE, 2016, pp. 582–597.
    [32] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” arXiv preprint arXiv:1706.06083, 2017.
    [33] F. Tramèr, A. Kurakin, N. Papernot, I. Goodfellow, D. Boneh, and P. McDaniel, “Ensemble adversarial training: Attacks and defenses,” arXiv preprint arXiv:1705.07204, 2017.
    [34] P. Cousot and R. Cousot, “Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints,” in Proceedings of the 4th ACM SIGACT-SIGPLAN symposium on Principles of programming languages, 1977, pp. 238–252.
    [35] J. Bertrane, P. Cousot, R. Cousot, J. Feret, L. Mauborgne, A. Miné, X. Rival, et al., “Static analysis and verification of aerospace software by abstract interpretation,” Foundations and Trends® in Programming Languages, vol. 2, no. 2-3, pp. 71–190, 2015.
    [36] X. Rival and K. Yi, Introduction to static analysis: an abstract interpretation perspective. Mit Press, 2020.
    [37] P. Cousot and R. Cousot, “Basic concepts of abstract interpretation,” in Building the Information Society. Springer, 2004, pp. 359–366.
    [38] L. Pulina and A. Tacchella, “An abstraction-refinement approach to verification of artificial neural networks,” in International Conference on Computer Aided Verification. Springer, 2010, pp. 243–257.
    [39] T. Gehr, M. Mirman, D. Drachsler-Cohen, P. Tsankov, S. Chaudhuri, and M. Vechev, “Ai2: Safety and robustness certification of neural networks with abstract interpretation,” in 2018 IEEE symposium on security and privacy (SP). IEEE, 2018, pp. 3–18.
    [40] G. Singh, T. Gehr, M. Mirman, M. Püschel, and M. Vechev, “Fast and effective robustness certification,” Advances in neural information processing systems, vol. 31, 2018.
    [41] M. Mirman, T. Gehr, and M. Vechev, “Differentiable abstract interpretation for provably robust neural networks,” in International Conference on Machine Learning. PMLR, 2018, pp. 3578–3586.
    [42] G. Singh, T. Gehr, M. Püschel, and M. Vechev, “An abstract domain for certifying neural networks,” Proceedings of the ACM on Programming Languages, vol. 3, no. POPL, pp. 1–30, 2019.
    [43] S. Kabir, R. U. Islam, M. S. Hossain, and K. Andersson, “An integrated approach of belief rule base and deep learning to predict air pollution,” Sensors, vol. 20, no. 7, p. 1956, 2020.
    [44] L. Chong, M. M. Abbas, A. M. Flintsch, and B. Higgs, “A rule-based neural network approach to model driver naturalistic behavior in traffic,” Transportation Research Part C: Emerging Technologies, vol. 32, pp. 207–223, 2013.
    [45] Z. Hu, X. Ma, Z. Liu, E. Hovy, and E. Xing, “Harnessing deep neural networks with logic rules,” arXiv preprint arXiv:1603.06318, 2016.
    [46] I. Harmon, S. Marconi, B. Weinstein, Y. Bai, D. Z. Wang, E. P. White, and S. Bohlman, “Improving rare tree species classification using domain knowledge,” 2022.
    [47] M. Kukar, I. Kononenko, et al., “Cost-sensitive learning with neural networks.” in ECAI, vol. 15, no. 27. Citeseer, 1998, pp. 88–94.
    [48] M. R. Hassan, B. Nath, and M. Kirley, “A fusion model of hmm, ann and ga for stock market forecasting,” Expert systems with Applications, vol. 33, no. 1, pp. 171–180, 2007.
    [49] E. Haber and M. Holtzman Gazit, “Model fusion and joint inversion,” Surveys in Geophysics, vol. 34, pp. 675–695, 2013.
    [50] P. Deepan and L. Sudha, “Fusion of deep learning models for improving classification accuracy of remote sensing images,” Journal of Mechanics of Continua and Mathematical Sciences, vol. 14, no. 1, pp. 189–201, 2019.
    [51] L. Breiman, “Bagging predictors,” Machine learning, vol. 24, pp. 123–140, 1996.
    [52] D. H. Wolpert, “Stacked generalization,” Neural networks, vol. 5, no. 2, pp. 241–259, 1992.
    [53] Y. Cao, Y. Lin, S. Ning, H. Pi, J. Zhang, and J. Hu, “Gan-based fusion adversarial training,” in Knowledge Science, Engineering and Management: 15th International Conference, KSEM 2022, Singapore, August 6–8, 2022, Proceedings, Part III.
    Springer, 2022, pp. 51–64.
    [54] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” arXiv preprint arXiv:1412.6572, 2014.
    [55] M. Andriushchenko, F. Croce, N. Flammarion, and M. Hein, “Square attack: a query-efficient black-box adversarial attack via random search,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIII. Springer, 2020, pp. 484-501.
    [56] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
    [57] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009, pp. 248-255.
    Description: 碩士
    國立政治大學
    資訊管理學系
    110356042
    Source URI: http://thesis.lib.nccu.edu.tw/record/#G0110356042
    Data Type: thesis
    Appears in Collections:[資訊管理學系] 學位論文

    Files in This Item:

    File Description SizeFormat
    604201.pdf3946KbAdobe PDF20View/Open


    All items in 政大典藏 are protected by copyright, with all rights reserved.


    社群 sharing

    著作權政策宣告 Copyright Announcement
    1.本網站之數位內容為國立政治大學所收錄之機構典藏,無償提供學術研究與公眾教育等公益性使用,惟仍請適度,合理使用本網站之內容,以尊重著作權人之權益。商業上之利用,則請先取得著作權人之授權。
    The digital content of this website is part of National Chengchi University Institutional Repository. It provides free access to academic research and public education for non-commercial use. Please utilize it in a proper and reasonable manner and respect the rights of copyright owners. For commercial use, please obtain authorization from the copyright owner in advance.

    2.本網站之製作,已盡力防止侵害著作權人之權益,如仍發現本網站之數位內容有侵害著作權人權益情事者,請權利人通知本網站維護人員(nccur@nccu.edu.tw),維護人員將立即採取移除該數位著作等補救措施。
    NCCU Institutional Repository is made to protect the interests of copyright owners. If you believe that any material on the website infringes copyright, please contact our staff(nccur@nccu.edu.tw). We will remove the work from the repository and investigate your claim.
    DSpace Software Copyright © 2002-2004  MIT &  Hewlett-Packard  /   Enhanced by   NTU Library IR team Copyright ©   - Feedback