Snapshot boosting: a fast ensemble framework for deep neural networks

查看全文 作　　者：Wentao [1,2]ZHANG;Jiawei [3]JIANG;Yingxia [4]SHAO;Bin [1,2,5]CUI

机构地区：[1]Center for Data Science,Peking University,Beijing 100871,China;[2]National Engineering Laboratory for Big Data Analysis and Applications,Beijing 100871,China;[3]Department of Computer Science,ETH Zurich,Zurich 8092,Switzerland;[4]Beijing Key Lab of Intelligent Telecommunications Software and Multimedia,School of Computer Science,Beijing University of Posts and Telecommunications,Beijing 100876,China;[5]Key Lab of High Confidence Software Technologies(MOE),Department of Computer Science,Peking University,Beijing 100871,China

出　　处：《Science China(Information Sciences)》索引2020年第63卷第1期,共12页

基　　金：supported by National Natural Science Foundation of China (Grant Nos. 61832001, 61702015, 61702016, 61572039);National Key Research and Development Program of China (Grant No. 2018YFB1004403);PKU-Tencent Joint Research Lab 摘　　要：Boosting has been proven to be effective in improving the generalization of machine learning models in many fields. It is capable of getting high-diversity base learners and getting an accurate ensemble model by combining a sufficient number of weak learners. However, it is rarely used in deep learning due to the high training budget of the neural network. Another method named snapshot ensemble can significantly reduce the training budget, but it is hard to balance the tradeoff between training costs and diversity. Inspired by the ideas of snapshot ensemble and boosting, we propose a method named snapshot boosting. A series of operations are performed to get many base models with high diversity and accuracy, such as the use of the validation set, the boosting-based training framework, and the effective ensemble strategy. Last, we evaluate our method on the computer vision(CV) and the natural language processing(NLP) tasks, and the results show that snapshot boosting can get a more balanced trade-off between training expenses and ensemble accuracy than other well-known ensemble methods. 关键词：ensemble learning deep learning BOOSTING neural network snapshot ensemble

相关文献

参考文献（37）
引证文献（4）
耦合文献（6）

参考文献(37)

引证文献(4)

耦合文献(6)

互联网出版许可证新出网证(渝)字10号全国400电话 - 免长途话费

Snapshot boosting: a fast ensemble framework for deep neural networks

参考文献(37)

引证文献(4)

耦合文献(6)

用户提示