| 1 | Reveal training performance mystery between Tensor Flow and PyTorch in the single GPU environment显示文摘Deep learning has gained tremendous success in various fields while training deep neural networks(DNNs)is very compute-intensive,which results in numerous deep learning frameworks that aim to offer better usability and higher performance to deep learning practitioners.Tensor Flow and Py Torch are the two most popular frameworks.Tensor Flow is more promising within the industry context,while Py Torch is more appealing in academia.However,these two frameworks differ much owing to the opposite design philosophy:static vs dynamic computation graph.Tensor Flow is regarded as being more performance-friendly as it has more opportunities to perform optimizations with the full view of the computation graph.However,there are also claims that Py Torch is faster than Tensor Flow sometimes,which confuses the end-users on the choice between them.In this paper,we carry out the analytical and experimental analysis to unravel the mystery of comparison in training speed on single-GPU between Tensor Flow and Py Torch.To ensure that our investigation is as comprehensive as possible,we carefully select seven popular neural networks,which cover computer vision,speech recognition,and natural language processing(NLP).The contributions of this work are two-fold.First,we conduct the detailed benchmarking experiments on Tensor Flow and Py Torch and analyze the reasons for their performance difference.This work provides the guidance for the end-users to choose between these two frameworks.Second,we identify some key factors that affect the performance,which can direct the end-users to write their models more efficiently. | Hulin DAI Xuan PENG Xuanhua SHI Ligang HE Qian XIONG Hai JIN | 2022 | Science China(Information Sciences)2022，65，1： | 6 |
您所在的机构暂未开通此部分的功能，请联系管理员