The Answer’s 42

One year and three months ago I wrote an article called AI or Nay? where I discussed my experiences, and overall came to the conclusion that AI can make your life better, but also to keep your feet on the ground.

Well, time has passed since then, and I’ve had quite a few experiences using these tools that have made me a bit wiser, I’d like to think. Let’s see how it all turned out…

Copilot Complete, the Bad Back Seat Driver

One of the things that came to irk me over the past year when working with Copilot was its code completion. Initially quite modest and helpful, I’m not 100% sure what happened but since August last year, it seems Copilot completion has come to believe its own hype about AI replacing software engineers as the main writer of code because now it will complete everything.

And not really in a helpful manner, either. I’ve turned it off for personal projects because while initially it used to offer thoughtful one line snippets, it now puts entire blocks of code in front of me, possibly stuff I don’t want.

Sometimes it’s been useful, but I find that constantly having something second guessing me and quite happily pasting in code if I happen to press tab has become highly annoying.

It’s gone from the helpful friend who’ll suggest something if they think you need help to a loudmouth that just talks over you and won’t let you get a word in edgeways. While Intellisense can be vaguely annoying and give inappropriate suggestions, Copilot complete now seems to prove an active disruptor to my mental model of what I want to do as not only do I need to assess the code I write, but also whatever nonsense the AI wants to burble at me.

After a year of using it on and off, I’ve found off is the better mode simply because I don’t want computer chatter constantly interrupting my flow of thought. Knowing what I’ve written, and what the computer has written is key.

Copilot Chat, Agent of Fortune!

Copilot Chat, on the other hand, off to the side in Visual Studio Code, is the exact opposite of the completions. It’s a helpful collaborator that I’ve found several uses for, especially the agent mode which has become one of my favourite things and two applications over the past year have stuck out in mind.

Problem One: CDK to YAML Conversion

I needed to translate a lot of TypeScript CDK code over to CloudFormation.

TypeScript CDK is great on many levels as it allows the developer to write their infrastructure-as-code (IAC) in a language that allows for them to define things programmatically in an easy to understand fashion (compared with the terse nature of YAMLs) and use standard TypeScript logic to deal with problems such as having something happen only in a certain environment.

This is on the surface much more agreeable, but under the surface CDK is generating YAMLs that are well, a nightmare, inserting names like 𝖬𝗒𝖲𝟥𝖡𝗎𝖼𝗄𝖾𝗍𝟤𝟥𝟦𝟧𝟥𝟧𝟥𝟧𝟦𝟥𝖥𝟢𝟢𝖠 or some other such nonsense as well as entire lambdas that you might not have been aware of until deployment, and maybe not even then.

If you are using CDK and having that be the end of the matter it’s fine, but if you’re going to swap CDK for CloudFormation for some reason, then you have to output and edit the nightmarish YAMLs that CDK has created, and well… they’re horrible to edit.

Enter Copilot Chat, which with the prompt “These YAMLs have been generated with CDK and contain many computer names that are not suited for humans. Please make this YAML suitable for human maintenance, removing all computer-generated names and structures and replacing them with human-friendly alternatives” got to work on the generated YAMLs and cleaned them up in the space of a few hours’ work.

The computer names vanished, and were replaced with much easier to understand terms like 𝖬𝗒𝖲𝟥𝖡𝗎𝖼𝗄𝖾𝗍, and after having had Copilot Chat’s agent mode do a once-over of it, what was left was much easier to get into shape ready for use.

Not using Copilot Chat probably would have meant at least an afternoon or more of painful editing, but the AI handled the computer-generated slop with aplomb!

Problem Two: Writing Tests

If you write a new routing, you have to write tests for it, otherwise you won’t know if it works and you won’t know if there are any edge cases. That’s just a fact of life in software engineering, but unless you are especially passionate about unit testing it can suck.

I’ve sat there before, trying to think up appropriate test data and sometimes it’s easy, but sometimes the tests have been quite difficult to write and I’m always nervous that there’s some case I’ve not thought of.

Once I figured out how to get Copilot Chat to do it, however, it’s made testing a lot easier. Giving it a prompt of “please write unit tests for my_new_function in newview.py, using XX test framework and covering all edge cases you can see” will often yield some pretty competent tests, which again make a task that could be time consuming much, much more painless.

The tests aren’t always correct, but often need tweaking and rewriting rather than being flatout wrong.

This makes them useful, and using AI for this has been quite beneficial as it’s rendered something that is very important, but also sometimes painful into a task that is now fairly easy to sort out and has added great value. I think this is what AI was made for!

First Make It Exist, Then Make It Good

And last but not least: it’s proven quite handy in a similarly irksome area of my personal projects – setting the damn things up.

I enjoy writing Vulkan renderers in C++ and Rust as a hobby: while I might not be a professional games developer, I still enjoy trying to replicate graphical tricks I see in games as the maths and code behind them fascinates me. However, one thing that I find incredibly annoying is the setup.

I’ve got the code for a basic Rust WinIt and C++ GLFW window committed to memory by now, but often to get around typing them out until I started using Copilot Chat I had to go and search through some of my repos from other projects, find the “blank window” repository I created, then paste that in, customise it and make sure it runs post-customization… which to be honest was a bit of a pain.

This has gone the way of the dodo because again, Copilot Chat does a pretty competent basic window for Vulkan and I don’t have to do the hunt-and-paste.

For instance, this is a window I created in Rust and Winit with the prompt: “Give me a Rust window that is suitable to display Vulkan using winit, and will disappear when I press escape” which is a perfectly good starting point for whatever I want to do with my Vulkan code. It may seem small, but this is where I’m seeing the valuable nature of LLM-assisted development coming through.

Good Vibes vs. Bad Vibes

I’ve expressed contempt for slopware: i.e. code written by non-technical people who’ve just got the LLM to dump out some random code, then run it and shipped it and declared to the world that they’ve got an awesome new product that everyone should use and pay for.

This is still bad, and will get the slop vendor owned by SQL injection attacks in no time and if they leak credit card information likely used. “We don’t need engineers, we can just get LLMs to do it!” is asking for trouble.

I think the initial euphoria has gone down though after several buggy, horrible web games were thrown onto the internet and the owners subsequently humiliated by the product in either sheer unusability or malicious attacks. The lesson was learned, but I don’t think we should discount vibe coding altogether.

If someone has taken the time to develop a deep knowledge of software, what practices make it good or bad, and have an eye for what they want then I would say such a person can be much more productive with vibe coding.

Unlike a complete neophyte, an experienced developer can get a lot of boilerplate out of the way and then take over from the machine to do more of what they need.

You can go from a “Hello World” Rust file to having a window with the Vulkan device, render passes and swapchain set up in 5 minutes or less, or go from a basic Django service to one that has everything you need ready to get going and put together something useful much, much quicker.

That is the real power of LLMs in my experience: as long as you give it an exact task, and set careful boundaries for it, and don’t expect it to write your entire software project for you, it can be a valuable collaborator and automate away much of the drudge work that can make development suck at times.

There are misfires such as Copilot completions: the babbling machine preempting me is not something I find useful, however Copilot chat, Claude Code and the overall field of agentic coding products have proven their worth to me.

Seeking Peaceful Coexistence

In the long run, I think that once the hype train has died down (and it will!), the creation of LLMs will have created a solid tool for software engineers to use to become more productive.

I’m not so sure about whether we’ll see the cornucopia of LLMs that we currently have sticking around: the field is becoming oversaturated and ripe for consolidation.

We’re also beginning to see local hardware such as AMD Ryzen CPUs shipping with onboard NPU cores as well to run LLMs and other forms of AI, and I think that security/privacy concerns and the increasing availability of good open source models such as Kimi-K2-Thinking by China’s Moonshot combined with the ability to run them locally will see LLMs becoming built-in, which addresses a lot of the concerns I had about corporate capture by OpenAI et al.

So overall my experiences over the past year with LLMs and AI have convinced me that peaceful coexistence is possible. There have been some misfires, but I’ve learned a lot about a powerful and valuable tool, and how we can integrate it into our work lives.