University of Washington

The University of Washington has introduced a more efficient large model tuning method called "proxy tuning". This method guides the prediction of the basic model by comparing the prediction results of a small tuned model and an untuned model, and achieves model tuning without touching the internal weights of the model. With the development of generative AI products such as ChatGPT, the parameters of the basic model continue to increase, so weight tuning requires a lot of time and computing power. To improve the tuning efficiency, this method can better retain the training knowledge during decoding while retaining the advantages of larger-scale pre-training. Researchers used the 13B and 70B original models of LlAMA-2...