top of page
Search

WILL AI MAKE US BETTER?

Writer's picture: Mikael SvanstromMikael Svanstrom
Feb 205 min read

I love AI. I love the fact that AI has been brought into the spotlight these past few years instead of hiding in the murky back alleys of data scientists and language specialists. Did we end up with questionable titles (I’m looking at you Mr. Prompt Engineer)? Yes. But all in all it has been a major step forward into public awareness.

So will it make us better at stuff? Of course! Humankind is on an ever-exponential direction towards bigger brains, greater thoughts and better tools. AI is just the latest and greatest in this development.

Unfortunately there has been research calling out that this may not always be the case. Depending on how we use AI we open ourselves up to unforced errors.


Business Consultants

In one piece of research, they measured how 758 consultants performed across 18 tasks for a fictional shoe company. One group used AI, the other did it the old-fashioned way. The AI-assisted group outperformed the no-AI group in almost every measure of performance.

This is great news of course, but in a specific task the opposite was true. This one task asked for strategic recommendations based on data from a spreadsheet and interviews with company insiders. However the spreadsheet had missing data. Only consultants who compared that data with the interviews could reach the correct conclusions. On this task, the AI-partnered consultants scored markedly worse, making 19% more mistakes than the other group.


Recruiters

Another study looked at professional recruiters and their job performance using various AI tools. 181 recruiters were hired to evaluate 44 job applications based on their math ability. The data came from an international test of adult skills, so the math scores were not obvious from the résumés.

The conclusion was that recruiters with higher-quality AI were worse than recruiters with lower-quality AI. They spent less time and effort on each résumé, and blindly followed the AI recommendations. They also did not improve over time. On the other hand, recruiters with lower-quality AI were more alert, more critical, and more independent. They improved their interaction with the AI and their own skills.

The result suggested that better AI didn’t lead to a better result. It led to lazier, more mistake prone recruiters.


Developers

Two studies in coding showed interesting results. In one of them there was no significant change in any key measurement. In terms of pull request (PR) cycle time (the time to merge code into a repository) and PR throughput (number of pull requests merged), Copilot neither helped nor hurt developers. The only aspect that changed was number of bugs, which increased with 41%.

In another, the number of pull requests increased with 26% suggesting more work had been completed, but that was the only measurement that changed significantly. For more senior developers it hardly changed at all.

It does make you wonder about the statements made some business leaders. Mark Zuckerberg says he will replace mid-level engineers soon.  Marc Benioff, the founder of Salesforce, is not hiring any more engineers in 2025, claiming a 30% boost in productivity. It is yet to be seen what this will mean in code quality and number of bugs.


Driving

I’m in the market for a fancy EV. One with so much technology and cleverness it is basically a smartphone on wheels. I love the idea that we will have self-driving cars, but right now we are in a strange shadowland of almost good enough. I did a drive with a friend of mine from Sydney to Melbourne and back. He has a Tesla Model 3 and on the freeway, it was absolutely amazing to be able to let it drive, with me as the passive overseer. It didn’t take long for me to worry less about the road and at times look at my travelling companion, much longer than I usually would. Like those scenes in movies where the driver looks at the person next to them for so long you are 100% sure and accident will happen. So if something did go wrong, something that the autopilot couldn’t handle. I’m pretty sure that would turn ugly real fast, because I wasn’t paying attention.

I’m going to venture into a bit of correlation vs causation here and I apologise to anyone statistically minded that will struggle with this next section. There has been a steady decline in motor vehicle accidents causing death. A steady development of safety features over the years have pushed down the number of deaths per capita across the world. But in the last few years there has been a change. In most high-income countries, the death rates have either plateaued or increased. I’m wondering if this is actually a result of the “smarter” safety features and half-autonomous systems, that lulls you into a false sense of security. Much the same as the other areas in this article, we become worse as the safety systems gets better.

A few sudden “step on the brake pedal” incidents, also known as phantom braking, did make me focus more on the road, but I doubt that was why they happened. I still don’t know why the car decided to step on the brake suddenly, on a highway with no cars around or any other obstacles.


An Author’s best friend?

I use many different LLM tools as part of my writing and creative process. It is a decent editor, a great research companion, a useful way to re-write a sentence when you get stuck, and to get a starting point of a summary. But I always fact check before using it. The more esoteric the question, the more likely errors creep in.


Conclusion

AI is amazing. It makes us better in so many ways. But it also lulls you into believing it will always spit out stellar work that can just be pasted into an email and send off as your own. AI is a tool for innovation and efficiency, not a replacement of the worker. At least not yet. And I think this is the case across the board in almost all fields. It is almost good enough, and we mistake that for always better than we are.

I asked AI to create a diagram that shows how, as AI gets better, people get stupider. It looked really cool, so I just used it with no changes. It sure is pretty…and nonsensical. I’m not ready to hand the torch to the AI overlords just yet.


References



 
 
 

Comentários

© 2024 by Mikael Svanström
bottom of page