Loading...
|
Please use this identifier to cite or link to this item:
https://nccur.lib.nccu.edu.tw/handle/140.119/130976
|
| Title: | Lasso迴歸於可詮釋預測分析：強階層與樹狀結構
Lasso Regression for Interpretable Predictive Analytics: Strong Hierarchy and Tree Structure |
| Authors: | 陳婷文
Chen, Ting-Wen |
| Contributors: | 莊皓鈞
周彥君
Chuang, Hao-Chun
Chou, Yen-Chun
陳婷文
Chen, Ting-Wen |
| Keywords: | 詮釋性
Lasso迴歸
機器學習
樹狀結構
強階層
Interpretability
Lasso regression
Machine learning
Tree hierarchy
Strong hierarchy |
| Date: | 2020 |
| Issue Date: | 2020-08-03 17:35:24 (UTC+8) |
| Abstract: | 有鑒於數據分析被廣泛應用在不同問題領域，且近年來資料筆數與變數數目大幅增加，以機器學習建構的預測模型因而興起，其中隨機森林和梯度提升機運用集成樹演算法，能在模型內納入自變數與依變數間的非線性關係並處理高維度資料，提升預測準確度。然而這類模型缺乏解釋性，在商業領域如金融授信風險評估難以使用，故產業界仍倚賴具高透通性的迴歸模型，但一般而言其預測準確度低於解釋性弱的集成式學習。本研究利用在高維建模相當重要的Lasso迴歸相關技術，探討兩個可大幅改善迴歸模型預測準確度並保留解釋性的方案，一為由Lim and Hastie (2015)提出運用自變數交互項拓展維度，但保留強階層使模型易解釋的Hierarchical group-lasso regularization，二為本研究提出的Cluster-while-regression with tree hierarchy，後者將樣本同步分群與訓練後產出數個迴歸模型，以分群加入非線性關係，結合樹狀結構與各子葉Lasso迴歸，以混合整數規劃進行訓練，達成模型的全域最佳化。接著以不同資料集比較以上所提到的五種演算法後，本研究運用的兩種強化版迴歸模型預測表現皆顯著優於Lasso迴歸，我們所提出的Cluster-while-regression with tree hierarchy預測準確度更不遜於隨機森林與梯度提升機，並保留高可解釋性，對可詮釋人工智慧有所貢獻。
Due to the availability of observational data and variables, predictive machine learning has been widely applied in different fields. Random Forests and Gradient Boosting Machine are two popular machine learning models which use ensemble trees to incorporate the nonlinear relationship between independent and dependent variables and to process high-dimensional data, resulting in improved prediction accuracy. However, these models are lack of interpretability and hence not applicable to business situations like credit risk assessment. As a results, practitioners still rely on the regression model for interpretability. To improve prediction accuracy, Lasso regression is a key technique to include high-dimensional data while avoiding overfitting. In this study, we discuss two Lasso-based models that can greatly improve prediction accuracy while retaining interpretability. One is Hierarchical group-lasso regularization, which was proposed by Lim and Hastie (2015) and uses interaction terms to expand the dimension and further enforces strong hierarchy to make the model easy to interpret. The other is Cluster-while-regression with tree hierarchy, which adds nonlinear relationships by clustering. This model simultaneously considers tree structure for clustering and runs Lasso regression for each cluster. A mixed-integer programming is applied to achieve global optimization of the model. These two enhanced Lasso regression models performs better than the traditional Lasso regression model in different datasets. Cluster-while-regression with tree hierarchy even performs not worse than Random Forests and Gradient Boosting Machine and at the same time retain high interpretability. Our study thus contributes to interpretable artificial intelligence. |
| Reference: | Alpaydin, E. (2020). Introduction to machine learning (4th ed.), America: MIT press.
Baardman, L., Levin, I., Perakis, G., & Singhvi, D. (2018). Leveraging comparables for new product sales forecasting. Production and Operations Management, 27(12), 2339-2449.
Bien, J., Taylor, J., & Tibshirani, R. (2013). A lasso for hierarchical interactions. Annals of statistics, 41(3), 1111.
Cox, D. R. (1984). Interaction. International Statistical Review/Revue Internationale de Statistique, 52(1), 1-24.
DeSarbo, W. S., Oliver, R. L., & Rangaswamy, A. (1989). A simulated annealing methodology for clusterwise linear regression. Psychometrika, 54(4), 707-736.
Dunn, J. W. (2018). Optimal trees for prediction and prescription (Doctoral dissertation, Massachusetts Institute of Technology, Massachusetts, America). Retrieved from http://dspace.mit.edu/handle/1721.1/7582
Farrar, D. E., & Glauber, R. R. (1967). Multicollinearity in regression analysis: the problem revisited. The Review of Economic and Statistics, 49(1), 92-107.
Friedman, J., Hastie, T., & Tibshirani, R. (2010). Regularization paths for generalized linear models via coordinate descent. Journal of statistical software, 33(1), 1.
Hoerl, A. E., & Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1), 55-67.
Hu, K., Acimovic, J., Erize, F., Thomas, D. J., & Van Mieghem, J. (2019). Forecasting new product life cycle curves: Practical approach and empirical analysis. Manufacturing and Service Operations Management, 21(1), 66-85.
Lim, M., & Hastie, T. (2015). Learning interactions via hierarchical group-lasso regularization. Journal of Computational and Graphical Statistics, 24(3), 627-654.
McCullagh, P., & Nelder, J. A. (1989). Generalized Linear Models(2nd ed.), America: CRC Press.
Ogutu, J. O., Schulz-Streeck, T., & Piepho, H. P. (2012). Genomic selection using regularized linear regression models: ridge regression, Lasso, elastic net and their extensions. BMC proceedings, 6, S10.
Park, Y. W., Jiang, Y., Klabjan, D., & Williams, L. (2017). Algorithms for generalized clusterwise linear regression. INFORMS Journal on Computing, 29(2), 301-317.
Russel, S., & Norvig, P. (2013). Artificial intelligence: a modern approach, America: Pearson Education Limited.
Späth, H. (1979). Algorithm 39 clusterwise linear regression. Computing, 22(4), 367-373.
Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1), 267-288.
Yang, L., Liu, S., Tsoka, S., & Papageorgiou, L. G. (2017). A regression tree approach using mathematical programming. Expert Systems with Applications, 78, 347-357.
Yuan, M., & Lin, Y. (2006). Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68(1), 49-67.
Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the royal statistical society: series B (statistical methodology), 67(2), 301-320. |
| Description: | 碩士
國立政治大學
資訊管理學系
107356008 |
| Source URI: | http://thesis.lib.nccu.edu.tw/record/#G0107356008 |
| Data Type: | thesis |
| DOI: | 10.6814/NCCU202001103 |
| Appears in Collections: | [資訊管理學系] 學位論文
|
Files in This Item:
| File |
Description |
Size | Format | |
|---|
| 600801.pdf | | 2131Kb | Adobe PDF2 | 1681 | View/Open |
|
All items in 政大典藏 are protected by copyright, with all rights reserved.
|
