政大機構典藏-National Chengchi University Institutional Repository(NCCUR):Item 140.119/155515

政大機構典藏-National Chengchi University Institutional Repository(NCCUR):Item 140.119/155515

English | 正體中文 | 简体中文 | Post-Print筆數 : 27 | 全文笔数/总笔数 : 114367/145401 (79%)
造访人次 : 53046234 在线人数 : 391

RC Version 6.0 © Powered By DSPACE, MIT. Enhanced by NTU Library IR team.

搜寻范围

查询小技巧：

您可在西文检索词汇前后加上"双引号"，以获取较精准的检索结果

若欲以作者姓名搜寻，建议至进阶搜寻限定作者字段，可获得较完整数据

进阶搜寻

主页 ‧ 登入 ‧ 上传 ‧ 说明 ‧ 关于政大典藏 ‧ 管理

到手机版

政大機構典藏 > 資訊學院 > 資訊科學系碩士在職專班 > 學位論文 > Item 140.119/155515

请使用永久网址来引用或连结此文件: https://nccur.lib.nccu.edu.tw/handle/140.119/155515

题名:	基於SARSA的5G場景下基站動態睡眠控制與能耗優化研究 Dynamic Sleep Control and Energy Optimization of 5G Base Stations Using SARSA Algorithm
作者:	李緒倫 Li, Hsu-Lun
贡献者:	張宏慶 Jang, Hung-Chin 李緒倫 Li, Hsu-Lun
关键词:	5G 基站能源優化睡眠模式服務品質強化學習 SARSA演算法 5G Base Station Energy Optimization Sleep Mode Quality of Service Reinforcement Learning SARSA Algorithm
日期:	2024
上传时间:	2025-02-04 16:11:29 (UTC+8)
摘要:	隨著5G網路技術的快速發展，行動網路需求增加使得基站數量大幅擴增，其能源消耗已成為行動網路營運的主要成本。根據研究顯示，基站的能源消耗約佔整個行動網路總能耗的60%至80%，且5G基站耗電量較4G增加2-3倍。本研究提出一個基於SARSA強化學習的自適應睡眠機制，以優化5G基站的能源效率。本研究透過系統化實驗確定了SARSA演算法的最佳參數組合(學習率α=0.01、折扣因子γ=0.9、探索率ε=0.05)，並設計動態權重獎勵函數w(L)，同時考慮能源消耗和服務品質(QoS)的平衡。該獎勵函數結合了睡眠模式權重(SM_weight)和緩衝區限制(Buf_lim)兩個關鍵參數，使系統能夠根據即時流量負載(L)和延遲要求，自適應地調整基站的睡眠策略。本系統採用Python實現SARSA強化學習模型的訓練與測試，並在四種典型流量模式（恆定、週期性、突發和隨機）下進行全面驗證。實驗結果表明，在20毫秒延遲閾值(Buflim)，低流量負載(5%)情況下，當SM_weight=1.0時，本系統可實現高達88.10%的節能效果。在不同的SM_weight配置下，系統均展現出優異的節能表現：當SM_weight=0.75時可達84.10%的節能，當SM_weight=0.5時可達63.10%的節能。在中等流量負載(20%)時，系統仍可維持可觀的節能效果，當SM_weight=1.0時可達49.70%的節能。即使在較高的業務負載(65%)下，系統仍可維持6.30%-8.50%的節能效果。隨著流量負載的進一步增加至80%-95%，系統仍能保持約5.60%-6.50%的穩定節能效果。實驗數據驗證了該系統在各種流量條件(5%-95%)和不同SM_weight配置(0-1.0)下的適應性與有效性，特別是在低負載情況下展現出極佳的節能潛力。本研究的創新點包括：(1)提出基於動態權重改進SARSA演算法；(2)設計自適應獎勵函數結構；(3)實現多等級睡眠模式的智能調控策略；(4)不同流量模式下的睡眠策略之權重分析。這些創新使系統能在確保服務品質的同時達到顯著的節能效果。未來研究著重於：(1)探索深度強化學習在複雜網路場景中的應用；(2)整合邊緣計算以優化資源配置；(3)研究6G網路在超大規模MIMO系統中的應用；(4)開發基於AI的預測模型以實現更主動的睡眠策略。 With the rapid development of 5G network technology, increased mobile network demands have led to a significant expansion in the number of base stations, making their energy consumption a major operational cost. Research shows that base stations account for 60% to 80% of total mobile network energy consumption, with 5G base stations consuming 2-3 times more power than 4G stations. This study proposes an adaptive sleep mechanism based on SARSA reinforcement learning to optimize 5G base station energy efficiency. Through systematic experiments, this study determined the optimal parameter combination for the SARSA algorithm (learning rate α=0.01, discount factor γ=0.9, exploration rate ε=0.05) and designed a dynamic weight reward function w(L) that balances energy consumption and Quality of Service (QoS). This reward function incorporates two key parameters: sleep mode weight (SM_weight) and buffer limit (Buf_lim), enabling the system to adaptively adjust base station sleep strategies based on real-time traffic load (L) and latency requirements. The system implements SARSA reinforcement learning model training and testing in Python and undergoes comprehensive validation under four typical traffic patterns (constant, periodic, burst, and random). Experimental results show that with a 20ms latency threshold (Buflim) under low traffic load (5%), the system can achieve up to 88.10% energy savings when SM_weight=1.0. The system demonstrates excellent energy-saving performance under different SM_weight configurations: 84.10% savings at SM_weight=0.75 and 63.10% savings at SM_weight=0.5. Under medium traffic load (20%), the system maintains considerable energy savings of 49.70% when SM_weight=1.0. Even under higher traffic loads (65%), the system maintains 6.30%-8.50% energy savings. As traffic load further increases to 80%-95%, the system maintains stable energy savings of about 5.60%-6.50%. The experimental data validates the system's adaptability and effectiveness under various traffic conditions (5%-95%) and different SM_weight configurations (0-1.0), showing particularly excellent energy-saving potential under low-load conditions. The innovations of this research include: (1) proposing a SARSA algorithm improved with dynamic weights; (2) designing an adaptive reward function structure; (3) implementing intelligent control strategies for multi-level sleep modes; (4) analyzing sleep strategy weights under different traffic patterns. These innovations enable significant energy savings while ensuring service quality. Future research focuses on: (1) exploring deep reinforcement learning applications in complex network scenarios; (2) integrating edge computing to optimize resource allocation; (3) studying 6G network applications in ultra-massive MIMO systems; (4) developing AI-based prediction models for more proactive sleep strategies.
參考文獻:	[1]W. Guo, N. M. F. Qureshi, I. F. Siddiqui and D. R. Shin, "Cooperative Communication Resource Allocation Strategies for 5G and Beyond Networks: A Review of Architecture, Challenges and Opportunities," Journal of King Saud University - Computer and Information Sciences, vol. 34, pp. 8054-8078, 2022. [2]A. Sufyan, K. B. Khan, O. A. Khashan, T. Mir and U. Mir, "From 5G to beyond 5G: A Comprehensive Survey of Wireless Network Evolution, Challenges, and Promising Technologies," Electronics, vol. 12, no. 10, pp. 2200, 2023. [3]M. A. Kamal, H. W. Raza, M. M. Alam, M. M. Su'ud and A. B. A. B. Sajak, "Resource Allocation Schemes for 5G Network: A Systematic Review," Sensors, vol. 21, no. 19, pp. 6588, 2021. [4]R. Tan, Y. Shi, Y. Fan, W. Zhu and T. Wu, "Energy Saving Technologies and Best Practices for 5G Radio Access Network," IEEE Access, vol. 10, pp. 51747-51756, 2022, doi: 10.1109/ACCESS.2022.3174089. [5]O. Shurdi, L. Ruci, A. Biberaj and G. Mesi, "5G Energy Efficiency Overview," European Scientific Journal, vol. 17, no. 3, pp. 315-327, 2021. [6]E. U. Udo, L. I. Oborkhale and C. C. Nwaogu, "Analysis and Evaluation of Energy Efficiency of 5G Networks in Wireless Communication," AZOJETE, vol. 19, no. 2, pp. 175-182, 2023. [7]S. Mishra, "Artificial Intelligence Assisted Enhanced Energy Efficient Model for Device-to-Device Communication in 5G Networks," Human-Centric Intelligent Systems, vol. 3, pp. 425-440, 2023. [8]L. Williams, B. K. Sovacool and T. J. Foxon, "The energy use implications of 5G: Reviewing whole network operational energy, embodied energy, and indirect effects," Renewable and Sustainable Energy Reviews, vol. 157, pp. 112033, 2022. [9] D. López-Pérez et al., "A Survey on 5G Radio Access Network Energy Efficiency: Massive MIMO, Lean Carrier Design, Sleep Modes, and Machine Learning," IEEE Communications Surveys & Tutorials, vol. 24, no. 1, pp. 652-697, 2022. [10]P. Kalita and D. Selvamuthu, "Stochastic modelling of sleeping strategy in 5G base station for energy efficiency," Telecommunication Systems, vol. 83, pp. 115-133, 2023. [11]A. Israr, Q. Yang and A. Israr, "Renewable microgeneration cooperation with base station sleeping-mode strategy for energy-efficient operation of 5G infrastructures," Sustainable Energy, Grids and Networks, vol. 38, pp. 101358, 2024. [12]S. Malta, P. Pinto and M. F. Veiga, "Using Reinforcement Learning to Reduce Energy Consumption of Ultra-Dense Networks With 5G Use Cases Requirements," IEEE Access, vol. 11, pp. 5417-5428, 2023, doi: 10.1109/ACCESS.2023.3236980. [13]M. Bala and M. Desai, "To Analyze the Maximized Energy Consumption by Algorithm Implementing Reinforcement Learning for Wireless Communication Technology," 2023 IEEE 4th Annual Flagship India Council International Subsections Conference (INDISCON), 2023, pp. 1-8, doi: 10.1109/INDISCON58499.2023.10270783. [14]M. Masoudi, M. G. Khafagy, E. Soroush, D. Giacomelli, S. Morosi and C. Cavdar, "Reinforcement Learning for Traffic-Adaptive Sleep Mode Management in 5G Networks," 2020 IEEE 31st Annual International Symposium on Personal, Indoor and Mobile Radio Communications, 2020, pp. 1-7, doi: 10.1109/PIMRC48278.2020.9217340. [15]P. Popovski, K. F. Trillingsgaard, O. Simeone and G. Durisi, "5G Wireless Network Slicing for eMBB, URLLC, and mMTC: A Communication-Theoretic View," IEEE Access, vol. 6, pp. 55765-55779, 2018, doi: 10.1109/ACCESS.2018.2872781. [16]P. Lähdekorpi, M. Hronec, P. Jolma and J. Moilanen, "Energy Efficiency of 5G Mobile Networks with Base Station Sleep Modes," 2017 IEEE Conference on Standards for Communications and Networking (CSCN), Helsinki, 2017, pp. 163-168, doi: 10.1109/CSCN.2017.8088631. [17]B. Debaillie, C. Desset, and F. Louagie, ‘‘A flexible and future-proof power model for cellular base stations,’’ in Proc. IEEE 81st Veh. Technol. Conf. (VTC Spring), May 2015, pp. 1–7. [18]M. Feng, S. Mao and T. Jiang, "Base Station ON-OFF Switching in 5G Wireless Networks: Approaches and Challenges," IEEE Wireless Communications, vol. 24, no. 4, pp. 46-54, August 2017, doi: 10.1109/MWC.2017.1600353. [19]F. E. Salem, A. Gati, Z. Altman, and T. Chahed, ‘‘Advanced sleep modes and their impact on flow-level performance of 5G networks,’’ in Proc.IEEE Veh. Technol. Conf., Sep. 2018, pp. 1–7. [20]Z. Umar, "Base Station Sleep Modes to Trade-off Energy Saving and Performance in 5G Networks," M.S. thesis, Dept. Electron. Telecommun., Politecnico di Torino, Turin, Italy, 2020. [21]J. Navarro-Ortiz, P. Romero-Diaz, S. Sendra, P. Ameigeiras, J. J. Ramos-Munoz, and J. M. Lopez-Soler, "A Survey on 5G Usage Scenarios and Traffic Models," IEEE Commun. Surv. Tutor., vol. 22, no. 2, pp. 905-929, 2020. [22]R. S. Sutton, "Reinforcement Learning: An Introduction," SIAM Review, vol. 63, no. 2, pp. 423-425, 2021. [23]D. Zhao et al., "Deep reinforcement learning with experience replay based on SARSA," in Proc. 2016 IEEE Int. Conf. Syst. Man Cybern., Budapest, Hungary, 2016, pp. 1-6. [24]H. Jiang et al., "An Improved Sarsa(λ) Reinforcement Learning Algorithm for Wireless Communication Systems," IEEE Access, vol. 7, pp. 115418-115427, 2019. [25]A. Gupta and R. K. Jha, "A Survey of 5G Network: Architecture and Emerging Technologies," IEEE Access, vol. 3, pp. 1206-1232, 2015. [26]M. Shariat, Ö. Bulakci, A. De Domenico, C. Mannweiler, M. Gramaglia, Q. Wei, A. Gopalasingham, E. Pateromichelakis, F. Moggio, D. Tsolkas, B. Gajic, M. Rates Crippa, and S. Khatibi, "A Flexible Network Architecture for 5G Systems," Wireless Commun. Mobile Computing, vol. 2019, Art. no. 5264012, Feb. 2019. [27]A. Taneja, N. Saluja, N. Taneja, A. Alqahtani, M. A. Elmagzoub, A. Shaikh, and D. Koundal, "Power Optimization Model for Energy Sustainability in 6G Wireless Network," Sustainability, vol. 14, no. 12, p. 7310, Jun. 2022. [28]T. P. Fowdur and B. Doorgakant, "A review of machine learning techniques for enhanced energy efficient 5G and 6G communications," Engineering Applications of Artificial Intelligence, vol. 122, p. 106032, 2023. [29]P. Kaur, R. Garg, and V. Kukreja, "Energy-efficiency schemes for base stations in 5G heterogeneous networks: a systematic literature review," Telecommunication Systems, vol. 84, pp. 115-151, 2023. [30]T. Rumeng, T. Wu, Y. Shi, and Y. Hu, "Intelligent Energy Saving Solution of 5G Base Station Based on Artificial Intelligence Technologies," in IEEE International Joint EMC/SI/PI and EMC Europe Symposium, 2021, pp. 739-742.
描述:	碩士國立政治大學資訊科學系碩士在職專班 104971015
資料來源:	http://thesis.lib.nccu.edu.tw/record/#G0104971015
数据类型:	thesis
显示于类别:	[資訊科學系碩士在職專班] 學位論文

文件中的档案:

档案	描述	大小	格式	浏览次数
101501.pdf		2533Kb	Adobe PDF	0	检视/开启

在政大典藏中所有的数据项都受到原著作权保护.

社群 sharing

著作權政策宣告 Copyright Announcement

1.本網站之數位內容為國立政治大學所收錄之機構典藏，無償提供學術研究與公眾教育等公益性使用，惟仍請適度，合理使用本網站之內容，以尊重著作權人之權益。商業上之利用，則請先取得著作權人之授權。
The digital content of this website is part of National Chengchi University Institutional Repository. It provides free access to academic research and public education for non-commercial use. Please utilize it in a proper and reasonable manner and respect the rights of copyright owners. For commercial use, please obtain authorization from the copyright owner in advance.

2.本網站之製作，已盡力防止侵害著作權人之權益，如仍發現本網站之數位內容有侵害著作權人權益情事者，請權利人通知本網站維護人員(nccur@nccu.edu.tw)，維護人員將立即採取移除該數位著作等補救措施。
NCCU Institutional Repository is made to protect the interests of copyright owners. If you believe that any material on the website infringes copyright, please contact our staff(nccur@nccu.edu.tw). We will remove the work from the repository and investigate your claim.

DSpace Software Copyright © 2002-2004 MIT & Hewlett-Packard / Enhanced by NTU Library IR team Copyright © - 回馈