Authors: Shen Li (Meta AI), Jessica Choi (Meta AI), Pavel Belevich (Meta AI), Yanli Zhao (Meta AI), Rohan Varma (Meta AI), Geeta Chauhan (Meta AI), Pritam Damania (Meta AI), Mahesh Yadav (Meta AI)
Collaborators: Vaibhav Singh (Google), Isaack Karanja (Google), Parvez Mulla (Google) Distributed training techniques advanced rapidly in recent…