Author of the publication

copy delete add this publication to your clipboard
community post
history of this post
URL
DOI
BibTeX
EndNote
APA
Chicago
DIN 1505
Harvard
MSOffice XML

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model.

S. Smith, M. Patwary, B. Norick, P. LeGresley, S. Rajbhandari, J. Casper, Z. Liu, S. Prabhumoye, G. Zerveas, V. Korthikanti, E. Zheng, R. Child, R. Aminabadi, J. Bernauer, X. Song, M. Shoeybi, Y. He, M. Houston, S. Tiwary, and B. Catanzaro. CoRR, (2022)

Please choose a person to relate this publication to

To differ between persons with the same name, the academic degree and the title of an important publication will be displayed. You can also use the button next to the name to display some publications already assigned to the person.

Shrija Rajbhandari

Sujan Rajbhandari

Meena Rajbhandari

Rajita Rajbhandari

Kishor Kumar Rajbhandari

Other publications of authors with the same name

Scalable and Efficient MoE Training for Multitask Multilingual Models.Y. Kim, A. Awan, A. Muzio, A. Cruz-Salinas, L. Lu, A. Hendy, S. Rajbhandari, Y. He, and H. Awadalla. CoRR, (2021)Fast LSTM Inference by Dynamic Decomposition on Cloud Systems.Y. You, Y. He, S. Rajbhandari, W. Wang, C. Hsieh, K. Keutzer, and J. Demmel. ICDM, page 748-757. IEEE, (2019)ZeRO-Offload: Democratizing Billion-Scale Model Training.J. Ren, S. Rajbhandari, R. Aminabadi, O. Ruwase, S. Yang, M. Zhang, D. Li, and Y. He. USENIX Annual Technical Conference, page 551-564. USENIX Association, (2021)SimiGrad: Fine-Grained Adaptive Batching for Large Scale Training using Gradient Similarity Measurement.H. Qin, S. Rajbhandari, O. Ruwase, F. Yan, L. Yang, and Y. He. NeurIPS, page 20531-20544. (2021)Optimizing the Four-Index Integral Transform Using Data Movement Lower Bounds Analysis.S. Rajbhandari, F. Rastello, K. Kowalski, S. Krishnamoorthy, and P. Sadayappan. PPoPP, page 327-340. ACM, (2017)Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model.S. Smith, M. Patwary, B. Norick, P. LeGresley, S. Rajbhandari, J. Casper, Z. Liu, S. Prabhumoye, G. Zerveas, V. Korthikanti and 10 other author(s). CoRR, (2022)DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via Multi-Modal Causal Attention.Z. Yao, X. Wu, C. Li, M. Zhang, H. Qin, O. Ruwase, A. Awan, S. Rajbhandari, and Y. He. CoRR, (2023)ZeRO: memory optimizations toward training trillion parameter models.S. Rajbhandari, J. Rasley, O. Ruwase, and Y. He. SC, page 20. IEEE/ACM, (2020)1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed.H. Tang, S. Gan, A. Awan, S. Rajbhandari, C. Li, X. Lian, J. Liu, C. Zhang, and Y. He. ICML, volume 139 of Proceedings of Machine Learning Research, page 10118-10129. PMLR, (2021)Accelerating Large Scale Deep Learning Inference through DeepCPU at Microsoft.M. Zhang, S. Rajbhandari, W. Wang, E. Zheng, O. Ruwase, J. Rasley, J. Li, J. Wang, and Y. He. OpML, page 5-7. USENIX Association, (2019)

BibSonomy

Disambiguation of "Rajbhandari, Samyam"

copy delete add this publication to your clipboard
community post
history of this post
URL
DOI
BibTeX
EndNote
APA
Chicago
DIN 1505
Harvard
MSOffice XML

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model.

Please choose a person to relate this publication to

Shrija Rajbhandari

Sujan Rajbhandari

Meena Rajbhandari

Rajita Rajbhandari

Kishor Kumar Rajbhandari

Other publications of authors with the same name

Disambiguation

BibSonomy

Disambiguation of "Rajbhandari, Samyam"

copydeleteadd this publication to your clipboardcommunity posthistory of this postURLDOIBibTeXEndNoteAPAChicagoDIN 1505HarvardMSOffice XML Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model.

Please choose a person to relate this publication to

Shrija Rajbhandari

Sujan Rajbhandari

Meena Rajbhandari

Rajita Rajbhandari

Kishor Kumar Rajbhandari

Other publications of authors with the same name

Disambiguation

copy delete add this publication to your clipboard
community post
history of this post
URL
DOI
BibTeX
EndNote
APA
Chicago
DIN 1505
Harvard
MSOffice XML

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model.