Just recently, a company called Cognition AI
released ‘the first software engineer AI’. It’s called Devin AI, and the news of it broke
the internet over its advanced capability. ‘But hey, can’t ChatGPT and other LLM models
already do coding? What’s special about Devin?’ First off, unlike other LLM models, Devin is
capable of learning how to use technologies that it was not familiar with. Like
in this demo shown Cognition Labs, Devin was able to look through a blog post to
learn how to generate images using ControlNet on Modal. This is a really great capability,
because it means that Devin doesn’t have to be trained on every library or documentation, it can
learn things on the go, just like any human could. Devin can also build and deploy apps from end
to end. As shown here, Devin was instructed to build a simple game called ‘The Game of
Life’.Devin broke down each steps necessary to create the game, and then proceed to give a
link so that the user can preview the result. It can also take incremental suggestions
about what should be improved on the game. While that in itself is already really good,
Devin is also capable of finding bugs in a codebase, and even address issues in Github
repositories. It sets up the environment, tried to reproduce the bug, and test the
code, all of that Devin did on its own. Cognition AI said that they evaluated Devin on
a SWE-bench, where Devin was tested to resolve real world GitHub issues. Devin showed
a vastly higher rate of issues resolved, at 13% compared to other LLMs like Claude,
Llama and ChatGPT, which don’t even break above 5%. If this is to be trusted, then that
means Cognition AI’s problem solving ability on programming is twice or even thrice than the other
competitions.To demonstrate how good Devin is, Cognition showed a demo in which Devin was
used to do a freelance hustle from Upwork, and it does the job just as well as if
you hired a human engineer to do it. And here’s the part that makes it really weird:
Devin is so advanced that, it can even train another AI! And it only needs the link to the
repository. Can you imagine that in a few years, AIs will just train other AIs to improve
upon themselves? This sounds insanely absurd, but assuming that this demo is valid, humans
might not be the ones creating AIs in the future. Now the company itself, Cognition AI, is founded
by Scott Wu, a math prodigy graduated from Harvard, and one of the best on competitive
programming, as he holds the title ‘Legendary Grandmaster’ on Codeforces. The team only consists
of 10 personnel, among which won gold medals in competitive programming as well, and previously
worked at companies such as Google Deepmind, Cursor, Scale AI, and other companies. They were
backed by Peter Thiel, who’s known for being a key investor in early stages of PayPal and Facebook,
and has announced a $21 million series A funding. With all of the capabilities Devin AI has, can
it really replace human programmers, and if so, what will happen to the job itself, and the
people currently employed in it? If AI can really program better than humans, should we
learn coding to prepare for the future? While these question cannot be answered outright at this
moment, it is very likely that whatever Devin AI, or other AI hasn’t been capable of, they will
be capable of it. Figures like Jensen Huang, the CEO of Nvidia, even suggested that coding
might not be a necessary skill in the future, as AI will close the gap between
programmers and the general public, making programming to be something that
everyone will be able to do naturally. What do you think? Is software engineering
jobs really over, or will Devin just be an icing on top of software companies? Tell
us what you think in the comment section.
