The foundation, current situation and future prospects of pre-training large language models

Abstract

The field of artificial intelligence has developed rapidly recently, and large language model technology, as a representative technology of it, can provide general knowledge and make many downstream tasks easier and more convenient. However, although many people use large language models to do some work, they still lack a systematically summarized literature. Therefore, in this article, we made a systematic summary. We first wrote about the early large language models, then we presented the development of GPT and how to use the GPT model, then we introduced the advanced GPT models, and finally we mentioned the risks and challenges faced by the GPT model. Our work can help users better use large language models.