1. Overview
Gen AI and tools like GitHub Copilot are not that good at replacing programmers for serious tasks, as of April 2025. It looks like we are still many months away from that stage. Many statements seen on the Internet are either marketing campaigns trying to sell existing AI tools, or too optimistic statements not based on experience with actual tools that exist today.
In this article, we are giving a summary of three articles on the Internet supporting the above statement, and the last one is this author’s personal development experience with GitHub Copilot.
2. Article 1: AI agents can’t reliably debug software.
Article 1: AI isn’t ready to replace human coders for debugging, researchers say
Summary:
- Even when given access to tools, AI agents can’t reliably debug software.
- .. (people) adjust their expectations because models aren’t good enough at the debugging part, and debugging occupies most of a developer’s time. That’s the suggestion of Microsoft Research.
- AI models… a far cry from what an experienced human developer can do
- …suggest some of the ambitious ideas about AI agents directly replacing developers are pretty far from reality
- … the models tend to produce code laden with bugs and security vulnerabilities, and they aren’t generally capable of fixing those problems
- .. the best outcome is an agent that saves a human developer a substantial amount of time, not one that can do everything they can do
3. Article 2: Under pressure to embrace AI, developers are growing frustrated
Article 2: AI coding mandates are driving developers to the brink
Summary:
- ..AI adoption is “tearing their company apart” as a rift emerges between leadership and the employees adopting such tools.
- For software developers specifically, there are concerns that AI coding tools are introducing errors into their code, failing at many tasks, and compounding technical debt.
- …developer faith in AI coding tools has quickly declined
- …developers describe a slew of technical issues and headaches associated with AI coding tools, from how they frequently suggest incorrect code and even delete existing code to the many issues they cause with deployments
- …the use of AI tools is causing an increase in incidents, with 68% of Harness respondents saying they spend more time resolving AI-related security vulnerabilities now compared to before they used AI coding tools
- GitHub Copilot… it was wrong as many times as it was right
- it takes a lot of time for someone (human) to review something they (AI tools) actually didn’t fully write and understand
- Executives mismanage their expectations …company leaders who don’t have close visibility into engineering workflows
- AI coding tools are still new, and figuring out how to use them effectively requires a concerted effort
4. Article 3: GitHub Copilot (GHC) can not be trusted with a bit complicated task
Article 3: GitHub Copilot (Gen-AI) is Helpful, But Not Great
Summary:
- GitHub Copilot (GHC)… Sometimes it is brilliant.. sometimes it wastes your time, especially because the verbose answers it gives are often off-topic
- It is useful sometimes, but only for local scope problems, and cannot see the bigger picture.
- …“personal feeling” is “it does not know it well”, it is “trying to guess it out”, and since it is a machine with a huge memory of millions of lines of code memorized, guesses are sometimes brilliant, sometimes off-topic.
- .. a huge disappointment that it can not get the C# syntax right all the time and check the existence of C# properties/methods by itself
- … it deleted the active line of code because a similar line of code was commented out
- I use “ghost test” from GHC a lot, review it, and accept suggestions when I like them.
- I use the GHC text-prompt page to ask for the generation of snippets or small functions with clear functionality…. No guarantees GHC will succeed here, but if it does, it can be brilliant.
- I no longer try bigger changes involving 3-4 files at the same time…Answers are at best incomplete, with many errors, like C# properties and methods that do not exist (it hallucinates?)
- … it is time and energy-consuming to REVIEW each answer GHC provides
- …it hallucinates about C# properties and methods that do not exist for well-known and frequently used APIs…
- … it generates more mess than useful code, for assigned pattern-based tasks involving 4 files.
- I ask GHC for help with problems, but read max 2 chat replies. Its answers tend to be verbose… If it does not give me a good answer in 2 attempts, I will go read Google for the same problem.
- GHC has a serious focus problem; answers are frequently off-topic.
- GitHub Copilot (GHC) can not be trusted with a bit complicated task involving several files at the same time. In such scenarios, results are incomplete and not time-efficient compared to direct manual programming.
- GitHub Copilot (GHC) tends to hallucinate about C# methods and properties that do not exist. The GHC-
- The generated code does not compile right away, requiring a lot of manual work to finish it.
5. Conclusion
The articles above suggest that some of the ambitious ideas about AI tools directly replacing developers are pretty far from reality.
Some authors believe it remains likely that the best outcome is an AI tool that saves a human developer a substantial amount of time, not one that can directly replace the developer.
6 References
[1] AI isn’t ready to replace human coders for debugging, researchers say
https://arstechnica.com/ai/2025/04/researchers-find-ai-is-pretty-bad-at-debugging-but-theyre-working-on-it/
[2] AI coding mandates are driving developers to the brink
https://leaddev.com/culture/ai-coding-mandates-are-driving-developers-to-the-brink
[3] GitHub Copilot (Gen-AI) is Helpful, But Not Great (March 2025)
GitHub Copilot (Gen-AI) is Helpful, But Not Great (March 2025)
