Replaced by AI?

Like many other people in the industry, I became wary of the news that AI was going to replace us started circulating in software professionals’ circles. By the time when people started fear-mongering and promising that in 6 months there will be no need in human developers, AI has already been around for a while. However it made a couple of significant leaps in features and efficiency right around the introduction of ChatGPT 3.5. All of a sudden people found themselves in several camps: some advocated for and preached the gospel of the coming of AI, some feared for their jobs and started looking for a way to secure their positions or hedge their bets, while others were skeptical about the hype that the industry was experiencing. I have started in the camp of those who thought that this wave might impact their livelihoods. Fun fact: I decided to start an MS in Cybersecurity for that reason - to secure against the uncertain future. Over time I transitioned into a camp of skeptics. To understand the proposition of the AI overlords and compare it to the reality, we need to examine what AI is truly good at and what it is at best subpar at.

AI-enabled Productivity

What is AI good at?

If you query any GPT like OpenAI ChatGPT, Google Gemini, or Anthropic Claude, they can give you some pretty impressive answers for a wide variety of questions. I have used these products to brainstorm ideas, outline papers, generate code, and even ask for a life advice. Most of these responses were pretty accurate and helpful. In particular when generating code, you can really lean into it and go fast. The context windows and the output size have been continually increasing and working with more at a time has become more of a reality. Using a tool like this could really speed up a lot of mundane tasks and assist in higher-level work like writing code.

What is AI bad at?

On the surface, it is very tempting to look at it and say that this is very human-like and approaching something approximating an Artificial General Intelligence (AGI). This has been the main source of hype lately - AI companies promising that AGI is right around the corner. However, the reality is pretty far from the truth. One of the things that I have worked on recently was a small minimal viable product (MVP) for a little checklist application that would help founders account for everything necessary in their product launches. While the start of the project was amazing - I had a working app with about a thousand lines of code in no time - the more the project grew the more the errors, bugs, and wrong implementation started to creep in.

When I tried to ask it to implement the drag & drop functionality using Alpine.js Sort plugin, it didn’t do the job. After apologizing and fixing the issue, nothing got fixed. The drag & drop didn’t get to see the light of day :(

When I tried to ask it to implement the drag & drop functionality using Alpine.js Sort plugin, it didn’t do the job. After apologizing and fixing the issue, nothing got fixed. The drag & drop didn’t get to see the light of day :(

One of the things with AI generation output is that there is always a margin of error that is present in the output also known as hallucination - when AI outputs something that is wrong or completely irrelevant. One of the things that occurred to me while working on the MVP was that the greater the code base grew - the more likely was the AI to make an error, produce a bug, or give an output that I was not looking for. That same margin of error given the larger input will produce a larger likelihood of an erroneous output. That is clearly a limitation that AI companies are not talking about. In addition to that, AI is still a predictive technology. The way GPT works is it takes all of the input that you provide and generates the word that it deems the most likely relevant to this context. After the word is generated, the entire input + that first word now serve as an input, which the model uses to evaluate the next most likely word. And it iterates like this for a while until it completes the output. This mode of operation cannot provide you with an answer whether something is correct or not. AI doesn’t know right from wrong. It knows likely from unlikely. And that’s about it. That is one of the major reasons why a lot of companies in industry sectors that operate with sensitive information or are bound by regulatory compliance are hesitant to adopt this new technology that can sometimes produce a faulty output.

How would you like to talk to a doctor that can sometimes tell you with a straight face that smoking certain amount of cigarettes per day is good for your health as a pregnant mother?

How would you like to talk to a doctor that can sometimes tell you with a straight face that smoking certain amount of cigarettes per day is good for your health as a pregnant mother?

Another interesting thing that was observed was the specificity of suggestions that the AI would present. For example, if you were to ask what tech stack you should use to create a web app, almost without failure it will recommend JS/TS/React/Node/Express stack. You know why? Because it is the most likely output. The majority of GitHub repos are written with the use of JS. It is obvious that the AI will suggest that as a solution. An expert would never just throw a suggestion at you because they understand that there are nuances to consider. Because of this focus on the most common stacks, if you were to ask it generate a solution in a technology for which there is far less input that went into the model’s training, it would likely give you a faulty output. In my experience, when asked to give a frontend made with Alpine.js and HTMX, instead of React or Vanilla JS, it produced some errors that had to be debugged after hours of researching and googling. Which ultimately begged the question: was the time that AI saved me by generating the code worth the code reviewing and debugging it on the tail end?

One other annoying aspect that I stumbled upon was the AI adding features that I did not request. I had to go back after multiple prompts and ask it to remove them.

One other annoying aspect that I stumbled upon was the AI adding features that I did not request. I had to go back after multiple prompts and ask it to remove them.

How AI impacts the amount of output

Most of the AI hype comes and is driven by this desire to produce more in less time. Speed at all costs. Gotta go fast. But we do not stop to think whether this is truly beneficial to businesses and customers. Long time ago our ability to truly understand the technology and have wisdom to discern how to apply it well and for the good of others has been outpaced by the speed at which the technology is evolving. There are too many examples to count where the technology has existed long before we realized its unintended consequences. Pausing and thinking has to happen first. AI is a prime example of this phenomena.

Where do we fit in?

Prompting

First and the most obvious place where we fit in (and where AI still needs us) is providing the input, querying, or prompting the models. Whether you are using them as a replacement for a search engine or bouncing off some ideas, we are the ones providing the input to the models. Even the agentic mode of AI still needs us to provide the request in the first place.

Seems trivial, right? Not exactly. In the realm of developing software for example, how do you know if you are asking the right questions? Is it sufficient to simply ask the AI to create an app for you that will track your running progress around your neighborhood? That depends. Do you want the app to use your location data? Do you want it to solely rely on accelerometer data? Do you want it to store the running data to a database? In which case, what database should it store it in? Do you want to analyze the data historically in graphs and charts? What solutions should you use for it?

These and many more other questions need to be carefully answered if you want to have a good working solution for the problem at hand. Can AI ask and self-prompt those questions in chain-of-thought reasoning? Certainly. But it will not know what answers it should ask in a given context based on the understanding of domain and requirements. It will yet again try to predict the answer that it deems most likely in the context of the question, which came from us, remember? Knowing what questions to ask and what prompts to give is essential and it comes from the experience of being a developer and having worked on projects, having failed and learned from failures.

Remember that asking AI to do something for you is equivalent to asking a cab driver to drive you to a destination. Except there is a catch. The cab driver doesn’t have a navigator. He just was trained on the most common routes. If you are asking to get to a known landmark, they will get you there no problem. But if you are asking them to drive you to your grandmother’s ranch out of town in an area where even Google Maps hasn’t mapped out a road yet, they will drive you to a totally different place from what you requested.

Understanding the results

After the model produces an output we have to decide whether the output is something of value and something that we want to keep. Did AI give me the code for the web app exactly how I wanted it to be? Do I need to re-prompt it because it created too many deviations? Or should I go ahead and modify the code myself? There are a lot of variables that decide whether the output is valuable or not. All of those variables are something that the AI model will not be able to decide on its own. AI companies are trying to create modes in which the model creates a chain-of-thought reasoning by reevaluating its own prompts and results. In the end it will only come to the middle of the road conclusion of what is the most likely output, just like it would do if we prompted it every step of the way. This leads me to a conclusion.

Value Judgement and Decision-making

The true differentiator between humans and AI (at the very least in their current GPT forms) is and always will be the ability to judge the value of information or a resource. Any expert in any industry has gained their expertise by trial and error, acquiring plethora of experience, and in the end, knowing what is right and what is wrong in a given context.

In my personal experience, over the last 8 years working in software development, some of my most valuable expertise comes from knowing what not to do in a given scenario. I worked in a startup where I was charged with choosing a tech stack. Looking back, some of the choices I made were not optimal, and I would choose differently if I had to work on a similar project today. Knowing what not to do shows me that a person has failed and learned from their mistake. That is more valuable than knowing all of the how-tos you can get your hands on.

Value judgement and decision making are here to stay. Trusting them to AI will lead to (and is already leading to) creation of buggy and vulnerable applications, and lots of other losses if we blindly hand over the keys to the AI.

Lack of context in AI systems

These systems lack context that real specialists keep in mind and understand deeply. On top of that, there are real technical limitations of context window. AI models can process and output only so much data. Real-world enterprise applications don’t even begin to fit into those boundaries, although I understand that this aspect will likely continue to be less and less of an issue if computational power will grow and training will become more potent. But even then, the models will still be trying to predict an output, rather than possess the knowledge or understanding of value of that output. Human experts will need to continually keep AI in check to make sure that no critical mistake is made.

Technology evolution

Like I mentioned above, new technology will require new training. Even with basic examples most of the models are unable to create a working Rust or Zig application due to small amount of training data that has been available thus far. The frontier of new tech will remain a domain where experts in the field will need to study it and make educated decision with it. AI will always lag behind, because it is dependent on the training data that humanity provides.

AI is a tool

AI is a tool. It is an impressive tool. It can enhance our workflows, provide us with more available data, and help with some mundane tasks. However, we have to approach it with understanding rather than blind hype that is motivated by AI evangelists and investors who throw money at anything that has word “AI” in it. We will continue to use the models and learn of new ways to apply it. What we should not do is think that these models will replace real valuable work and expert understanding any time soon.

At the time of this writing OpenAI had 185 vacant engineering positions available to apply to.

At the time of this writing OpenAI had 185 vacant engineering positions available to apply to.

Google Careers had 2446 open positions for engineering worldwide.

Google Careers had 2446 open positions for engineering worldwide.

Anthropic was hiring for about 90 different engineering positions.

Anthropic was hiring for about 90 different engineering positions.

Remember that OpenAI, Google, and Anthropic are still dependent on the real software developers despite having promised us that “The end is nigh” and that their AI is an equivalent of a legion of seasoned developers. If that were true, the Great Replacement would have already happened, and they would have been done with their workforce a while ago. Therefore, until they put their money where their mouths are, I don’t think we have to fear being replaced by AI.

AI is here to stay. And it will absolutely change and evolve the landscape of our workflows. Staying committed to learning and adapting to the changing landscape and keeping our tools and our knowledge sharp will give us a stable platform to continue being in demand in our fields of work. I furthermore believe that the wave of AI vibecoded applications will create a tsunami of buggy and vulnerable applications for which real developers and security specialists will be asked to come and fix the mess.

Stay sharp and up-to-date in your expertise. This will allow you to weather any major change in the industry and stay relevant in your field. Keep learning!