GPT-2

GPT-2 is a model launched by OpenAI in 2019. Its training cost was once $256 per hour. So, five years later, in the GPT-4 era, with the advancement of software, hardware and data, will the time and cost required to train the same model be reduced? The answer is yes. According to Tom's Hardware today, Andrej Karpathy, former director of Tesla AI, co-founder of OpenAI and project developer, used llm.c to "reproduce" GPT-2.