GitHub copilot PR review is useful

On GitHub pull requests, there’s the option to ask their LLM for a code review. It’s available to GitHub Pro users (US$48/year at the time of writing) and in organizations ($19/user/month).

It has limited language support today. I’ve been using it against TypeScript and HTML/React, and I’ve found it useful. It has made some performance and accessibility suggestions.

What it does

From GitHub’s documentation:

When a user requests a code review from Copilot, Copilot scans through the code changes, plus additional relevant context, and provides feedback on the code. As part of that feedback, it may also provide specific suggested code changes.

It looks like this

Many of the times I’ve used this, Copilot has summarised the change without suggestion. When it does make a suggestion it looks like this:

Note that there’s an offer to “Commit suggestion”.

I don’t accept commit suggestions

I use the suggestion as an idea for a change, and then I go to understand what it means and work my own change. I do this because I want to make sure it’s not bullshit, but also because of uncertainty over the origin of the suggestion, and that it may suggest something that infringes — even though I have  blocked matches to public code (see later).

The T&Cs here say:

The code, functions, and other output returned to you by GitHub Copilot are called “Suggestions.” GitHub does not own Suggestions. You retain ownership of Your Code and you retain responsibility for Suggestions you include in Your Code. It is entirely your decision whether to use Suggestions generated by GitHub Copilot. If you use Suggestions, GitHub strongly recommends that you have reasonable policies and practices in place designed to prevent the use of a Suggestion in a way that may violate the rights of others.

Privacy

There are controls for how your code, and the code of others, is used:

Conclusions

I was already a Pro user, so it’s worth it for me to try this out. I’m not sure I would pay extra for this just by itself.

This feature doesn’t replace a human pull request, but it’s a useful quick sanity check before asking a person to review. It gives you the chance to improve the PR, and sometimes learn a little bit about a language you might be unfamiliar with.