政大機構典藏-National Chengchi University Institutional Repository(NCCUR):Item 140.119/149470

政大機構典藏-National Chengchi University Institutional Repository(NCCUR):Item 140.119/149470

English | 正體中文 | 简体中文 | Post-Print筆數 : 27 | Items with full text/Total items : 113324/144300 (79%)
Visitors : 51119026 Online Users : 800

RC Version 6.0 © Powered By DSPACE, MIT. Enhanced by NTU Library IR team.

Scope

please add "double quotation mark" for query phrases to get precise results

please goto advance search for comprehansive author search

Adv. Search

Home ‧ Login ‧ Upload ‧ Help ‧ About ‧ Administer

Goto mobile version

政大典藏 > College of Commerce > Department of MIS > Theses > Item 140.119/149470

Please use this identifier to cite or link to this item: https://nccur.lib.nccu.edu.tw/handle/140.119/149470

Title:	程式預測預訓練模型的攻擊與防禦研究 Natural Attack and Defense for Pre-trained Models of Code Analysis
Authors:	黃舜硯 Huang, Shun-Yen
Contributors:	郁方 Yu, Fang 黃舜硯 Huang, Shun-Yen
Keywords:	預訓練模型自然攻擊側錄 CodeBERT Natural Attack Profile
Date:	2024
Issue Date:	2024-02-01 10:56:56 (UTC+8)
Abstract:	預訓練程式碼分析模型透過惡意程式碼偵測等應用徹底改變了軟體工程。然而，它們的有效性受到了對抗性攻擊的威脅，例如 ALERT，它透過巧妙地修改輸入來操縱模型輸出。這篇論文提出了一種基於側錄的方法來識別針對 CodeBERT 的 ALERT 攻擊。我們利用動態程式追蹤來捕獲模型在處理原始樣本和對抗性樣本時的內部行為。這些追蹤記錄捕獲了關於函式調用的詳細資訊，包括它們的呼叫次數、返回值和執行時間。透過仔細比較這些追蹤記錄，我們希望識別出 ALERT攻擊的存在與否。此外，我們利用神經網路進行訓練。該神經網路的訓練集分別正常的程式及惡意程式，其中正常有被攻擊及沒被攻擊過的，惡意程式亦然。我們訓練結果如下：在正常程式程式資料集中，兩種模型提取屬性實現了 62% 和 72.2% 的準確率，在惡意程式資料集中，兩種模型提取屬性實現了 70% 和 89.1% 的準確率，在混合程式資料集中，兩種模型提取屬性實現了 69.3% 和 71.6% 的準確率。這些發現證明了基於效能分析的技術在預訓練程式碼模型中偵測對抗性攻擊的潛力。這項研究為進一步探索和改進這些方法開闢了道路，最終有助於預訓練模型在關鍵軟體工程任務中的彈性提升。 Pre-trained code analysis models have revolutionized software engineering with applications like malicious code detection. However, their effectiveness is threatened by adversarial attacks like ALERT, which subtly alter inputs to manipulate model outputs. This paper presents a novel tracing-based approach to identify ALERT attacks targeting CodeBERT. We leverage dynamic program tracing to capture the model's internal behavior while processing both original and adversarial samples. These traces capture detailed information about function calls, including their counts, return values, and execution times. By meticulously comparing these traces, we aim to identify characteristic patterns indicative of ALERT manipulations, revealing the attack's presence. Further, we explore the use of a neural network trained on profiled data categorized as normal, malicious, and mixed. Our investigation yielded promising results: two key model attributes derived from the traces achieved an accuracy of 62% and 72.2% on normal code, 70% and 89.1% on malicious code, and 69.3% and 71.6% on the combined dataset. These findings demonstrate the potential of profiling-based techniques for detecting adversarial attacks in pre-trained code models. This research opens avenues for further exploration and refinement of such methods, ultimately contributing to the resilience of pre-trained models in critical software engineering tasks.
Reference:	[1] Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang, and M. Zhou, “CodeBERT: A pre-trained model for programming and natural languages,” in Findings of the Association for Computational Linguistics: EMNLP 2020. Online: Association for Computational Linguistics, Nov. 2020, pp. 1536–1547. [Online]. Available: https://aclanthology.org/2020.findings-emnlp.139 [2] B. Zhang and M. Becker, “Variability code analysis using the vital tool,” in Proceedings of the 6th International Workshop on Feature- Oriented Software Development, ser. FOSD ’14. New York, NY, USA: Association for Computing Machinery, 2014, p. 17–22. [Online]. Available: https://doi.org/10.1145/2660190.2662113 [3] T. Kamiya, “Introducing parameter sensitivity to dynamic code-clone anal- ysis methods,” in 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), vol. 3, 2016, pp. 19–20. [4] K. Cho, B. van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, “Learning phrase representations us- ing rnn encoder-decoder for statistical machine translation,” 2014, cite arxiv:1406.1078Comment: EMNLP 2014. [Online]. Available: http://arxiv.org/abs/1406.1078 [5] U. Alon, O. Levy, and E. Yahav, “code2seq: Generating se- quences from structured representations of code,” in International Conference on Learning Representations, 2019. [Online]. Available: https://openreview.net/forum?id=H1gKYo09tX [6] K. Clark, M.-T. Luong, Q. V. Le, and C. D. Manning, “Electra: Pre-training text encoders as discriminators rather than generators,” in International Conference on Learning Representations, 2020. [Online]. Available: https://openreview.net/forum?id=r1xMH1BtvB [7] Z. Yang, J. Shi, J. He, and D. Lo, “Natural attack for pre-trained models of code,” in Proceedings of the 44th International Conference on Software Engineering, ser. ICSE ’22. New York, NY, USA: Association for Computing Machinery, 2022, p. 1482–1493. [Online]. Available: https://doi.org/10.1145/3510003.3510146 [8] Z. Dai, S. Liu, Q. Li, and K. Tang, “Saliency attack: Towards imperceptible black-box adversarial attack,” ACM Trans. Intell. Syst. Technol., vol. 14, no. 3, apr 2023. [Online]. Available: https://doi.org/10.1145/3582563 [9] H. Hussain, T. Duricic, E. Lex, D. Helic, M. Strohmaier, and R. Kern, “Structack: Structure-based adversarial attacks on graph neural networks,” in Proceedings of the 32nd ACM Conference on Hypertext and Social Media, ser. HT ’21. New York, NY, USA: Association for Computing Machinery, 2021, p. 111–120. [Online]. Available: https://doi.org/10.1145/3465336.3475110 [10] D. Z ̈ugner, O. Borchert, A. Akbarnejad, and S. G ̈unnemann, “Adversarial attacks on graph neural networks: Perturbations and their patterns,” ACM Trans. Knowl. Discov. Data, vol. 14, no. 5, jun 2020. [Online]. Available: https://doi.org/10.1145/3394520 [11] T. Sonnekalb, B. Gruner, C.-A. Brust, and P. M ̈ader, “Generalizability of code clone detection on codebert,” in Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering, ser. ASE ’22. New York, NY, USA: Association for Computing Machinery, 2023. [Online]. Available: https://doi.org/10.1145/3551349.3561165 [12] X. Meng, J. M. Anderson, J. Mellor-Crummey, M. W. Krentel, B. P. Miller, and S. Milakovi ́c, “Parallel binary code analysis,” in Proceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, ser. PPoPP ’21. New York, NY, USA: Association for Computing Machinery, 2021, p. 76–89. [Online]. Available: https://doi.org/10.1145/3437801.3441604 [13] J. Obert and T. Loffredo, “Efficient binary static code data flow analy- sis using unsupervised learning,” in 2021 4th International Conference on Artificial Intelligence for Industries (AI4I), 2021, pp. 89–90. [14] S. Sargsyan, V. Vardanyan, H. Aslanyan, M. Harutunyan, M. Mehrabyan, K. Sargsyan, H. Hovahannisyan, H. Movsisyan, J. Hakobyan, and S. Kur- mangaleev, “Genes isp: code analysis platform,” in 2020 Ivannikov Ispras Open Conference (ISPRAS), 2020, pp. 35–39. [15] Q. Ashfaq, R. Khan, and S. Farooq, “A comparative analysis of static code analysis tools that check java code adherence to java coding standards,” in 2019 2nd International Conference on Communication, Computing and Digital systems (C-CODE), 2019, pp. 98–103. [16] R. Paramitha and Y. D. W. Asnar, “Static code analysis tool for laravel framework based web application,” in 2021 International Conference on Data and Software Engineering (ICoDSE), 2021, pp. 1–6. [17] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Minneapolis, Minnesota: Association for Computational Linguistics, Jun. 2019, pp. 4171–4186. [Online]. Available: https://aclanthology.org/N19-1423 [18] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert- Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, “Language models are few-shot learners,” in Proceedings of the 34th International Conference on Neural Information Processing Systems, ser. NIPS’20. Red Hook, NY, USA: Curran Associates Inc., 2020. [19] S. Black, L. Gao, P. Wang, C. Leahy, and S. Biderman, “GPT-Neo: Large scale autoregressive language modeling with meshtensorflow,” Oct. 2021. [Online]. Available: https://doi.org/10.5281/zenodo.5551208 [20] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” J. Mach. Learn. Res., vol. 21, no. 1, jan 2020. [21] M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, “BART: Denoising sequence-to- sequence pre-training for natural language generation, translation, and comprehension,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Online: Association for Computational Linguistics, Jul. 2020, pp. 7871–7880. [Online]. Available: https://aclanthology.org/2020.acl-main.703 [22] M. A. Umer, C. M. Ahmed, M. T. Jilani, and A. P. Mathur, “Attack rules: An adversarial approach to generate attacks for industrial control systems using machine learning,” in Proceedings of the 2th Workshop on CPSIoT Security and Privacy, ser. CPSIoTSec ’21. New York, NY, USA: Association for Computing Machinery, 2021, p. 35–40. [Online]. Available: https://doi.org/10.1145/3462633.3483976 [23] J. Mu, B. Wang, Q. Li, K. Sun, M. Xu, and Z. Liu, “A hard label black-box adversarial attack against graph neural networks,” in Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, ser. CCS ’21. New York, NY, USA: Association for Computing Machinery, 2021, p. 108–125. [Online]. Available: https://doi.org/10.1145/3460120.3484796 [24] B. Wang and N. Z. Gong, “Attacking graph-based classification via manipulating the graph structure,” in Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, ser. CCS ’19. New York, NY, USA: Association for Computing Machinery, 2019, p. 2023–2040. [Online]. Available: https://doi.org/10.1145/3319535.3354206 [25] X. Lin, C. Zhou, H. Yang, J. Wu, H. Wang, Y. Cao, and B. Wang, “Ex- ploratory adversarial attacks on graph neural networks,” in 2020 IEEE International Conference on Data Mining (ICDM), 2020, pp. 1136–1141. [26] C. Guo, “An overview of adversarial sample attacks and defenses for graph neural networks,” in 2021 International Conference on Intelligent Comput- ing, Automation and Applications (ICAA), 2021, pp. 252–260. [27] Y. Xu, X. Wei, P. Dai, and X. Cao, “A2sc: Adversarial attacks on subspace clustering,” ACM Trans. Multimedia Comput. Commun. Appl., mar 2023, just Accepted. [Online]. Available: https://doi.org/10.1145/3587097 [28] S. Zhou, C. Liu, D. Ye, T. Zhu, W. Zhou, and P. S. Yu, “Adversarial attacks and defenses in deep learning: From a perspective of cybersecurity,” ACM Comput. Surv., vol. 55, no. 8, dec 2022. [Online]. Available: https://doi.org/10.1145/3547330 [29] M. D’Ambros, M. Lanza, and R. Robbes, “Evaluating defect prediction approaches: A benchmark and an extensive comparison,” Empirical Softw. Engg., vol. 17, no. 4–5, p. 531–577, aug 2012. [Online]. Available: https://doi.org/10.1007/s10664-011-9173-9 [30] T. Hall, S. Beecham, D. Bowes, D. Gray, and S. Counsell, “A systematic literature review on fault prediction performance in software engineering,” IEEE Trans. Softw. Eng., vol. 38, no. 6, p. 1276–1304, nov 2012. [Online]. Available: https://doi.org/10.1109/TSE.2011.103 [31] X. Yang, D. Lo, X. Xia, Y. Zhang, and J. Sun, “Deep learning for just-in-time defect prediction,” in Proceedings of the 2015 IEEE International Conference on Software Quality, Reliability and Security, ser. QRS ’15. USA: IEEE Computer Society, 2015, p. 17–26. [Online]. Available: https://doi.org/10.1109/QRS.2015.14 [32] S. Wang, T. Liu, and L. Tan, “Automatically learning semantic features for defect prediction,” in Proceedings of the 38th International Conference on Software Engineering, ser. ICSE ’16. New York, NY, USA: Association for Computing Machinery, 2016, p. 297–308. [Online]. Available: https://doi.org/10.1145/2884781.2884804 [33] Y. Shin and L. Williams, “An empirical model to predict security vulnerabilities using code complexity metrics,” in Proceedings of the Second ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, ser. ESEM ’08. New York, NY, USA: Association for Computing Machinery, 2008, p. 315–317. [Online]. Available: https://doi.org/10.1145/1414004.1414065 [34] I. Chowdhury and M. Zulkernine, “Using complexity, coupling, and cohesion metrics as early indicators of vulnerabilities,” Journal of Systems Architecture, vol. 57, no. 3, pp. 294–313, 2011, special Issue on Security and Dependability Assurance of Software Architectures. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1383762110000615 [35] Y. Shin, A. Meneely, L. Williams, and J. A. Osborne, “Evaluating com- plexity, code churn, and developer activity metrics as indicators of software vulnerabilities,” IEEE Transactions on Software Engineering, vol. 37, no. 6, pp. 772–787, 2011. [36] G. Apruzzese, M. Andreolini, L. Ferretti, M. Marchetti, and M. Colajanni, “Modeling realistic adversarial attacks against network intrusion detection systems,” Digital Threats, vol. 3, no. 3, feb 2022. [Online]. Available: https://doi.org/10.1145/3469659 [37] B. Li, J. Xu, S. Wu, S. Ding, J. Li, and F. Huang, “Detecting adversarial patch attacks through global-local consistency,” in Proceedings of the 1st In- ternational Workshop on Adversarial Learning for Multimedia, ser. ADVM ’21. New York, NY, USA: Association for Computing Machinery, 2021, p. 35–41. [Online]. Available: https://doi.org/10.1145/3475724.3483606 [38] J. Chen, H. Xu, J. Wang, Q. Xuan, and X. Zhang, “Adversarial detection on graph structured data,” in Proceedings of the 2020 Workshop on Privacy-Preserving Machine Learning in Practice, ser. PPMLP’20. New York, NY, USA: Association for Computing Machinery, 2020, p. 37–41. [Online]. Available: https://doi.org/10.1145/3411501.3419424 [39] C. Ferrari, F. Becattini, L. Galteri, and A. D. Bimbo, “(compress and restore)n: A robust defense against adversarial attacks on image classification,” ACM Trans. Multimedia Comput. Commun. Appl., vol. 19, no. 1s, jan 2023. [Online]. Available: https://doi.org/10.1145/3524619 [40] N. Liu, M. Du, R. Guo, H. Liu, and X. Hu, “Adversarial attacks and defenses: An interpretation perspective,” SIGKDD Explor. Newsl., vol. 23, no. 1, p. 86–99, may 2021. [Online]. Available: https://doi.org/10.1145/3468507.3468519 [41] A. Pattanaik, Z. Tang, S. Liu, G. Bommannan, and G. Chowdhary, “Ro- bust deep reinforcement learning with adversarial attacks,” in Proceedings of the 17th International Conference on Autonomous Agents and MultiA- gent Systems, ser. AAMAS ’18. Richland, SC: International Foundation for Autonomous Agents and Multiagent Systems, 2018, p. 2040–2042. [42] I. Rosenberg, A. Shabtai, Y. Elovici, and L. Rokach, “Adversarial machine learning attacks and defense methods in the cyber security domain,” ACM Comput. Surv., vol. 54, no. 5, may 2021. [Online]. Available: https://doi.org/10.1145/3453158 [43] C. Zhang, Z. Wang, R. Mangal, M. Fredrikson, L. Jia, and C. Pasareanu, “Transfer attacks and defenses for large language models on coding tasks,” 2023. [44] C. D. Manning, P. Raghavan, and H. Sch ̈utze, Introduction to Information Retrieval. Cambridge, UK: Cambridge University Press, 2008. [Online]. Available: http://nlp.stanford.edu/IR-book/information- retrieval-book.html [45] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems, 2017, pp. 5998–6008. [46] R. Agarwal, “Explaining bert simply using sketches,” Apr 2021. [Online]. Available: https://mlwhiz.medium.com/explaining-bert-simply- using-sketches-ba30f6f0c8cb [47] K. Clark, M.-T. Luong, Q. V. Le, and C. D. Manning, “Electra: Pre-training text encoders as discriminators rather than generators,” in 47 International Conference on Learning Representations, 2020. [Online]. Available: https://openreview.net/forum?id=r1xMH1BtvB [48] Y. Zhou, S. Liu, J. Siow, X. Du, and Y. Liu, Devign: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks. Red Hook, NY, USA: Curran Associates Inc., 2019. [49] P. He, B. Li, X. Liu, J. Chen, and Y. Ma, “An empirical study on software defect prediction with a simplified metric set,” Information and Software Technology, vol. 59, pp. 170–190, mar 2015. [Online]. Available: https://doi.org/10.10162Fj.infsof.2014.11.006 [50] J. Demˇsar, T. Curk, A. Erjavec, ˇCrt Gorup, T. Hoˇcevar, M. Milutinoviˇc, M. Moˇzina, M. Polajnar, M. Toplak, A. Stariˇc, M. ˇStajdohar, L. Umek, L. ˇZagar, J. ˇZbontar, M. ˇZitnik, and B. Zupan, “Orange: Data mining toolbox in python,” Journal of Machine Learning Research, vol. 14, pp. 2349–2353, 2013. [Online]. Available: http://jmlr.org/papers/v14/demsar13a.html
Description:	碩士國立政治大學資訊管理學系 110356040
Source URI:	http://thesis.lib.nccu.edu.tw/record/#G0110356040
Data Type:	thesis
Appears in Collections:	[Department of MIS] Theses

Files in This Item:

File	Description	Size	Format
604001.pdf		4058Kb	Adobe PDF	0	View/Open

All items in 政大典藏 are protected by copyright, with all rights reserved.

社群 sharing

著作權政策宣告 Copyright Announcement

1.本網站之數位內容為國立政治大學所收錄之機構典藏，無償提供學術研究與公眾教育等公益性使用，惟仍請適度，合理使用本網站之內容，以尊重著作權人之權益。商業上之利用，則請先取得著作權人之授權。
The digital content of this website is part of National Chengchi University Institutional Repository. It provides free access to academic research and public education for non-commercial use. Please utilize it in a proper and reasonable manner and respect the rights of copyright owners. For commercial use, please obtain authorization from the copyright owner in advance.

2.本網站之製作，已盡力防止侵害著作權人之權益，如仍發現本網站之數位內容有侵害著作權人權益情事者，請權利人通知本網站維護人員(nccur@nccu.edu.tw)，維護人員將立即採取移除該數位著作等補救措施。
NCCU Institutional Repository is made to protect the interests of copyright owners. If you believe that any material on the website infringes copyright, please contact our staff(nccur@nccu.edu.tw). We will remove the work from the repository and investigate your claim.

DSpace Software Copyright © 2002-2004 MIT & Hewlett-Packard / Enhanced by NTU Library IR team Copyright © - Feedback