AI Startups Cognition Launched a new AI programmerGenie", its performance was amazing, instantly defeating Devin and GPT-4 to become the world's most powerful AI programming assistant.
This AI programmer scored as high as 30.08% on the authoritative testing platform SWE-Bench, far exceeding Devin's 13.8% and Swe-agent+GPT-4's 12.47%.
You may be curious, how did Genie do it? As early as December 2022, Alistair Pullen, co-founder of Genie, presented this project at the University of London. He hopes to create an AI program that can automatically code, debug, and optimize like humans. After more than a year of development, Genie finally entered the testing phase and received $2.5 million in seed round financing.
Alistair mentioned that Genie's success is closely related to its training data and methods. Unlike traditional large model fine-tuning, Genie uses a special data set that contains the reasoning process of human programmers. These data cover the gradual discovery of knowledge and case-based decision-making processes, enabling Genie to demonstrate judgment similar to that of human engineers when facing complex problems.
In addition, Genie also uses a unique "self-improvement mechanism". Initially, Genie was trained on high-quality data to achieve a "perfect" state, but in the process, Genie made mistakes in its own judgments and improvements. To overcome this problem, the developers used Genie to generate some synthetic data to further enrich the training content. This is like a mother teaching her child to walk and giving correct guidance after each fall.
After multiple iterations of training, Genie’s capabilities have improved significantly, and it can even show creative solutions to problems it has never seen before.In terms of functions, Genie supports a variety of development tasks, including function development, bug fixing, code refactoring, code testing, etc., covering dozens of programming languages such as JavaScript, Python, and Java.
Now, Genie is open for trial applications. You can register through the official website, and testing permissions are expected to be issued in the next few weeks.
Official blog: https://cosine.sh/blog/state-of-the-art
Experience address: https://cosine.sh/register