DEV’in
Seems like new AI tool drops every day. The big developer buzz recently has been around Devin by Cognition which claims to be the first AI software engineer.
Hi, I’m Brian and I’ve been a software developer at small to medium sized tech companies for over 20 years. I’ll give you my perspective and call out a couple of valuable threads around AI.
What Does It Mean?
Small shop developers are always looking for leg up when it comes to software development. We are quick, and often free, to try out new tools to increase productivity. The catch is usually a free trial and decent community or commercial support. We try tools, if they work, we keep them and if they don’t pan out, we abandon them quickly.
When the press release for Devin came over the wire, I was skeptical. I use Microsoft CoPilot and SonarQube, and they do a decent job but not to the degree that I’d consider them an extra team member. Was Devin really that different?
Turns out it isn’t. It’s better than generic LLMs at generating or fixing code, but it’s still well below the skill level of the most junior dev. While some breathlessly proclaim that Devin would eliminate developer jobs others commented that the Devin announcement was more to generate buzz and to different the tool from established players like CoPilot and Cody.
I haven’t tried Devin, and not many people have. It is intentionally obscured behind a very carefully crafted demo that, much like social media, seeks to cast its image in the best light. Bottom line is that there isn’t much real-world testing of the model, so it is too soon to gauge its potential, unlike the existing heavyweights.
DHH Says Things
David Heinemeier Hansson ruffled a few feathers when he suggested that automated development tools, like Devin, would eliminate a lot of jobs. He even implied that recent layoffs were a result, which is preposterous. I commented on the thread that the reset in tech is due to the end of ZIRP and other staffing excesses. AI tools are in the infancy of adoption and any company that claims to be eliminating jobs because of it is selling something.
I would concede that the long-term impact of coding automation is twofold. First it will be a huge productivity boost for engineers, especially experienced once. But on the flip side the first coding jobs to be "replaced" will be entry level tasks like bug fixes. This will impact junior developers more and will make the field of software engineering less accessible to new folks.
Daily.Dev’s Watercooler
The watercooler convo asked: Devin: The end of Developers... for 5 years or so?
Sure it made a splash, but how is it different than other "co-pilot tools"? It's probably not, or at least that is the consensus of folks that I read. Beneath the hype is probably another company that is trying to different itself from Microsoft and Sourcegraph. Gergely Orosz said as much in "The Pulse #85" and a number of folks on the Daily.Dev discussion thread agree.
Welcome to the Team?
Pragmatic Engineer asked: “What Changed in 50 Years of Computing?”
A lot, particularly the composition of a software development team. One role that jumped out at me was the "language lawyer", opinionated expert in the intricacies of a coding language. I suggested that AI tools would be very good at evaluating code quality, identifying security issues and enforcing style standards. Then again we already have tools that can do this, AI will make them better.
Who Needs a Search Engine?
In a tangentially related thread You aren't upset enough about the war on hyperlinks by Adam Singer I pointed out that AI driven search will only make the problem worse.
The interconnected nature of the web used to result in search engines crawling content to discover new content. To combat spam sites began to limit cross-linking or outright banning it. The result was that the Internet became a lot less connected.
Google drives most organic search traffic and if recent changes are any indicator the last hyperlink driven discovery mechanism is about to die. As the company tweaks the algorithm to favor AI generated results it's likely that SEO is dead and hyperlinks count for nothing. The focus shifts to the training data sets used for models.
But I wonder: Where does this training data come from? Who compiles it and how do they rank results? Perhaps SEO and hyperlinks aren't dead after all.
Bottom Line
If you are excited about new AI powered tools, GREAT! It’s hard to not be enthusiastic about the direction tech is headed. Those of us who have lived through a few cycles know that predicting the future is fun but hard to get right. It’s even harder to build and to break through the bottlenecks. Or as the theory says it’s more difficult to train the monkey than build the pedestal.
The point is that we are a long way from automation disrupting software development. Highly trained engineers are still required and will be for some time. But every software engineer must be proficient with AI tools. Moreover, we need to be smart enough to identify when AI hallucinates or gets it wrong. That means understanding design and patterns, but also the context of the problem to be solved.