Revolutionizing Machine Learning: Techniques and Discoveries.
Researchers at MIT have developed a technique to "grow" larger machine-learning models from smaller ones, reducing the time and cost required to train them. The method, called a learned Linear Growth Operator (LiGO), uses machine learning to learn a linear mapping of the parameters of the smaller model, enabling faster training of the larger model. The technique saves about 50% of the computational cost required to train a large model, compared to methods that train a new model from scratch, and the models trained using the MIT method performed as well as, or better than, models trained with other techniques.