In recent years, large language models (LLMs) have demonstrated remarkable performance on code-related tasks, and various code LLMs have emerged one after another.
These success stories indicate that pre-training on large-scale code data can significantly enhance a model’s core programming capabilities.
In addition to pre-training, some methods that post-train LLMs using instruction data have also achieved significant improvements in understanding instructions and the quality of responses.
