NvidiaRecently, we have collaborated with Hugging Face and ServiceNow to release a series of LLMs models called StarCoder2.It hopes to become a new standard in the field of code generation, with many advantages such as performance, transparency and cost-effectiveness.
The series of models includes a 3 billion parameter model trained by ServiceNow, a 7 billion parameter model trained by Hugging Face, and a 15 billion parameter model trained by NVIDIA.
This was achieved using a new code dataset called Stack v2, which is seven times larger than Stack v1; new training techniques also mean the model can better understand low-resource programming languages like COBOL, mathematics, and program source code discussions.
StarCoder2 has been trained in 619 programming languages and can perform professional tasks such as source code generation, workflow generation, and text summarization. Nvidia said that developers can use it for code completion, advanced code summarization, code snippet retrieval, etc., thereby improving work efficiency.
NVIDIA said that compared with the initial version of StarCoder LLMs, the new 3 billion parameter model further streamlined and screened high-quality parameters, and its performance is equivalent to the initial version of StarCoder with 15 billion parameter models.
StarCoder2 is licensed under the BigCode Open RAIL-M license, which allows royalty-free access and use. Interested users can download it from the BigCode project GitHub The source code of the page can be obtained from Hugging Face Download the model.