Using GitHub Copilot for writing documentation

Over the past few months, I’ve had the chance to dive into GitHub Copilot, and I must say, my experience with it from a programming perspective has been alright. However, what really caught my attention is how it can come in handy when it comes to writing documentation.

Let’s be honest, documenting a project can be a real drag. I’ve found that Copilot can help me out by suggesting sentences and paragraphs based on the context of what I’m writing.

I want to mention as a disclaimer that I haven’t used Copilot for writing documentation for any code projects yet. I’ve only used it for writing documentation for our internal processes and tools. I’ve also used it to write this blog post, but more on that later.

LLMs and documentation

When it comes to large language models (LLMs) and documentation, there are at least two ways that these generative models can be used. In this post, I’ll be focusing on the first approach, which uses the LLM to assist in writing documentation.

The other way is to make the AI understand existing documentation and then use that to generate explanations for questions that the users might have. I’m eagerly waiting for access to the Copilot for Docs beta to see how well it performs.

In this case, someone still needs to write the documentation, but the users can get answers to their questions without having to read through the whole thing. I find this to be a very welcome development as it can help make learning new tools and technologies more accessible.

But let’s get back to the topic of this post.

The good

I’ve found that Copilot can be very helpful when writing documentation. You can use it word by word as a kind of autocomplete, or you can insert whole sentences when the suggestions are good enough. The Ctrl+Left and Tab shortcuts are your friends here.

Copilot usually has some idea what I’m trying to say and suggests a few options. I can then pick the one that I like the most and continue writing. You can cycle through the suggestions with Ctrl+[ and Ctrl+], or use the Ctrl+Enter shortcut to synthesize a list of suggestions.

I also like how it is able to adapt to what I’m writing and suggests relevant content. For example, if I’m writing about a service that has separate environments for development, staging, and production, it will suggest reasonable completions for each environment. This is something that I’ve found to be very useful when writing documentation for services running in Azure as it is usually able to suggest the correct resource group names and other details.

On the other hand, it doesn’t come in your way as you can just ignore the suggestions if you don’t like them. Sometimes the results can become a bit repetitive, but as long as you are in control, it’s not that big of a deal.

Actually, probably the best part is that it can quite often put in the right words for you when you’re not sure how to phrase something. This is especially useful when you’re writing in a language that is not your native language, like I am.

What is actually a little surprising is that sometimes it seems to be able to link to the official documentation of the thing that you’re writing about. For example, when I was documenting how we use the Azure Kubernetes Service and mentioned the connection to Azure Container Registry, it suggested a working link to the official documentation that instructs how to set up the connection.

Overall, I think Copilot is a bit like IntelliSense in your IDE. Similarly, the code suggestions can sometimes be crappy, but quite often they are just what you need. There just isn’t a standard library for writing text, so it’s not like you can just import a package and get all the documentation you need.

The bad

While Copilot can be helpful, it has its downsides.

One minor inconvenience with Copilot is that it doesn’t seem to be able to suggest text in a middle of a paragraph. So, if you want to add something in the middle, you need to start a new paragraph and then merge them together to get the suggestion.

Also sometimes there just isn’t enough context for Copilot to suggest anything useful. This is especially true when you’re writing about something that is not very common.

Copilot can’t know the reasons behind the decisions that you’ve made, so those explanations fall on you. Even if it has access to the code, it can’t know why you’ve decided to do something in a certain way.

And even though I said that the repetitive suggestions are not a big deal, it can get a bit annoying sometimes and maybe even distract you from what you’re trying to say.

The suggestions can also be quite similar to each other so there’s not much variety. It would be nice if it could suggest a few different options instead of having the same suggestion with slightly different wording.

What about blogging?

I’ve been using Copilot as an aid to write this blog post and it has been a mixed experience. It has been able to suggest some sentences and paragraphs that match perfectly, but it typically doesn’t know what I’m trying to say until the rest of the paragraph is obvious. In those cases, it does save a few keystrokes, though.

Still, it’s kind of interesting to see what it suggests. Sometimes it even surprises me with something that I didn’t think of myself.

Also, as I mentioned earlier, it is able to help with phrasing things. And because it knows the common idioms, it can suggest better ways to say something. The problem with this though is that the suggestion depends on what you’ve already written and it can’t fix those earlier parts.

Maybe the subject of this post is a bit too meta for Copilot to understand. (Or maybe it’s just not very good at writing blog posts.) ((Or maybe I’m just not very good at writing blog posts.)) (((Everything after the first six words of this paragraph was written by Copilot, so take that as you will.)))

I would really like to like it. When it works, it almost feels like magic. But when it doesn’t, it’s a little annoying.

Anyhow, I think I will have to keep using Copilot for a while to see if it can actually help me with writing blog posts.

Conclusion

I guess my point is that Copilot is not a silver bullet for writing documentation, but that doesn’t make it useless. I can only suggest you try it out and keep it in your toolbox if you find it useful.